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Spidey 


ack before Google was born, 
Ee and even longer before it 

became a verb, the World Wide 
Web was often searched by a little 
spider on a surfboard. Webcrawler was 
to many of us in the mid 1990s what 
Google is to the entire planet now. Of 
course, now | use Google to search 
the Internet, but in its day, that little 
spider was the gateway to knowledge. 
Times have changed, and the Internet 
has grown. Likewise, Web development 
in general has changed drastically 
during the past 20 years. Gone are the 
blink tags and “under construction” 
animated GIFs. Gone is Geocities. 
Even the giant AOL is a shadow of its 
former self. This month, we look at 
Web development as it is today, built 
on the best Web development platform 
available, which in our opinion, is Linux. 

Our own Katherine Druckman starts 

the issue with a look at Drupal and what 
to expect with Drupal 8—specifically from 
the perspective of Drupal community 
members from all over the world. It’s no 
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Remembering 


SHAWN POWERS 


secret Katherine loves Drupal, and she’s 
able to share with us the views of kindred 
spirits. Feeling right at home himself this 
issue, Reuven M. Lerner teaches how to 
use Watir, a tool for browser-testing Ruby 
code without the need to start up and 
navigate the various browsers manually. 
If you test your code (and you should), 
Watir is worth checking out. 

Dave Taylor continues his series 
on Cribbage this month, and turns a 
game | have always called “that one 
with the pegs” into an interesting and 
fairly complex script. If you like math, 
you'll fall in love with Dave's version of 
Cribbage. Kyle Rankin deals with Pi this 
month, but unlike Dave, Kyle’s Pi is of 
the raspberry variety. He returns to his 
beer fridge, and makes it both more 
efficient and more modern. We're sure 
there are things the Raspberry Pi can’t 
do, but so far, they elude us. 

Even | get into the spirit of Web 
development a bit this month. 
Granted my contribution is about 
three lines of PHP, but it’s an integral 


part of my column on problem solving 
with Linux. My dynamic DNS service 
decided to delete my account one 
day, so | decided to use Linux tools 
and scripting hacks to fix the problem 
on my own. If you need a primer on 
thinking outside the box, | guarantee 
my method doesn’t exist in any box. 

Alexander Castillo describes all the 
things a front-end developer should 
know about Drupal. Every version of 
Drupal brings new features, and for 
the last few releases, the learning 
curve has been declining steadily. 
Whether you’re creating entire Drupal 
themes or just want to customize some 
CSS, Alexander's article is invaluable. 

Developing code on your own dev 
box is a noble tradition and one that 
works well for many environments. 
Once the environment grows to 
include multiple developers, multiple 
environments for test and production 
and multiple locations, keeping 
things similar can be a nightmare. 

Ben Hosmer looks at Salt Stack and 
Vagrant this month. Although the two 
might sound like pirate names, they're 
actually a set of applications that can 
keep your development environments 
similar, with very little effort. 

If you think JavaScript has had its day 
in the sun, and it’s time for something 
newer, you're in luck. This month, James 
Slocum demos a completely different 


CURRENT_ISSUE.TAR.GZ 


tool for interactive Web programs—Dart. 
The language Is from Google, which 
implies it’s not a fly-by-night idea. Dart 
is still very new, but the idea is exciting. 
James explains how it works and even 
gives a demonstration of it in action. 
The Internet and the World Wide 
Web aren‘t going away any time soon. 
In fact, as Web sites become more 
and more complex, scaling to meet 
demand Is a challenge. Pablo Graziano 
shows how to use Varnish as a reverse- 
caching proxy to speed up server 
response time and help scale heavy 
loads. Pablo walks through setting up, 
configuring and tweaking your system 
to squeeze every bit of performance 
possible out of your Web servers. 
When Spidey the surfboard-riding 
Webcrawler was first introduced, it 
indexed a whopping 4,000 Web sites. 
The Web has grown significantly since 
then, and thanks to Web developers, its 
usefulness has grown as well. Whether 
your first search engine was Webcrawler 
or you were born after Google became 
a verb, this issue should be interesting. | 
know we liked putting it together.m 


Shawn Powers is the Associate Editor for Linux Journal. 
He’s also the Gadget Guy for LinuxJournal.com, and he has 
an interesting collection of vintage Garfield coffee mugs. 
Don’t let his silly hairdo fool you, he’s a pretty ordinary guy 
and can be reached via e-mail at shawn @linuxjournal.com. 
Or, swing by the #linuxjournal IRC channel on Freenode.net. 
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letter in the 
mea December 
2012 issue: 
whitespace 
is allowed in Fortran variable 
names, as is in Fortran keywords. 
It is simply ignored. Here's a 
small example: 


Project: Build a 
vi-Fi Honeypo 


BEST PRACTICES WORKING PLUS: MORE 
FOR CREATING WITH WEB SHELL SCRIPT 
PASSWORDS SOCKETS GAMES 


hoel@pchoel:~> cat 1j224.f 
programtl1lj22 4 
Wiehe dT te Cr) 
end 

hoel@pchoel:~> ifort 1j224.f 

hoel@pchoel:~> ./a.out 

Hello LinuxJournal 
hoel@pchoel:~> gfortran 1j224.f 
hoel@pchoel:~> ./a.out 


Hello LinuxJournal 
—Berthold Hollmann 


Dave Taylor replies: | find it 
awesome that we’re still talking 
about Fortran. Now, what about 
Algol-68? 
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Readers’ Choice Awards 2012 

| would really like to see LibreOffice 
as a contender for next year. It should 
have had its own slot for the 2012 
awards. It has been around and doing 
great on its own for some time now. 
Thanks for your consideration. As you 
noted in the article, many even wrote 
in LibreOffice. 

—Fran Parker 


| agree. It’s funny, the difficult part 
every year is coming up with the 
correct options!—Ed. 


Digital LJ and Dropbox Sync 

| love the digital version of the 
magazine! But, | would like to suggest 
you make the digital versions available 
via Dropbox sync, much like O'Reilly 
and Pragmatic Bookshelf do with their 
digital books. | find myself logging in 
to the LJ site and downloading the 
PDF versions for safe-keeping into a 
Dropbox folder (I already get LJ on my 
iPad mini via Newsstand). It would be 
awesome if this was automatic with 
Dropbox sync! 


Keep up the great work! 
—Terry Dunlap 


That's an interesting idea, Terry. | do the 


same thing, actually. I’m not sure if it is 
easy with our current distribution model, 
but | like the idea. We'll look into it!—Ed. 


January 2013 Work the Shell 
I've been a Linux user and reader of 
Linux Journal since the beginning, 
and | always find something valuable 
in every issue of your magazine. One 
of my favourite columns is Work the 
Shell, because I’m always looking for 
neat ways to process files and data 
in different ways. Often—no offense, 
Dave—it is the letters from readers 
that provide that aha moment! 


This time (regarding Dave Taylor’s 
column the January 2013 issue), | 
have a comment of my own. When 
attempting to shuffle a deck of cards, 
why not just use one of the well-known 
shuffle algorithms? Then, there is no 
need to worry about picking a card for 
the second (or third) time on reaching 
the end of the deck. Here is a piece of 
code for the Fisher-Yates algorithm, 
courtesy of Wikipedia: 


#!/bin/bash 
Tor ((i=O, 15925 1+*)) 


do 
deck [$i] =$i 


, LETTERS | 


done 

for ((i=51; 
do 

rand=$(( $RANDOM % (it+1) )) 
tmp=${deck[i]} deck[i]=${deck[rand] } 
deck[rand]=$tmp 

done 

echo ${deck[@] } 


ie, a> 2) 


I'm sure this can be improved, but you 
get the idea. 
—dik harris 


Dave Taylor replies: Thanks for 

your note and sample code. It’s ironic 
that when | was in college studying 
for my degree in computer science | 
complained about having to re-invent 
the wheel when Knuth’s The Art of 
Computer Programming was on my 
shelf and had tons of great algorithms. 
Zoom forward, and now I’m re-inventing 
the wheel in every column rather than 
looking for optimal results. 


But here’s the thing: while the code you 
sent me might be a good “shuffle”, it’s 
completely obfuscated, and since my 
primary goal with the Work the Shell 
column is to explain and explore the 
concepts underlying good script writing, 
using a 12-line block of complex code 
seems counter-productive. 
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Ill simply suggest we publish your 
letter and my response and see how 
the rest of the reader community feels 
about the subject. 


RPi 

I've been thinking of getting a Raspberry 
Pi to make a digital video-capture device 
for our wood-turning chapter (BAW). We 
currently use an old tape video camera. 
Then we have to transfer from tape to 
a PC, edit the video and produce it. 
We have two small cameras attached 
to tripods and can switch back and 
forth between them to show the 
demo on a 46" flatscreen so everyone 
watching the demo, up to 40 people, 
can see it better. But, when making a 
video, we have only the one camera! 
We sure would like to improve the 
process. What do you think? 

—Rex 


Shawn Powers replies: | like the idea 
of using a little Pi device, but | worry 
it might not have the processing 
horsepower to process and store 
video. You might consider a couple 
decent USB Webcams with long cables 
connected to a single beefy computer. 
Either solution sounds more fun than 
video tape, however, so / think you're 
headed in the right direction! 
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WiFi Analyzer 

| have a picture of Tux alongside the 
Android robot on the outside of my 
cubicle. The going joke in the office is that 
| can analyze the Wi-Fi with my phone, but 
my co-engineers cannot because they all 
have 10S. Another good app is WiFi Map 
Maker by Dave McKellar. It really helped 
me figure out which neighbor is flooding 
my house with his Wi-Fi signal. Thanks 
for the article and good work. 
—Roman 


Shawn Powers replies: | love that 
app too! I’ve never had a good 
enough GPS signal indoors to use 
it very well. Your scenario sounds 
perfect. I’m glad it worked for you. 


Memorable Password Generator 
After reading this XKCD comic about 
developing easy-to-remember, hard-to- 
crack passwords: http://xkcd.com/936, 
| stumbled upon this page for manually 
generating a strong password with 
six-sided die: http://world.std.com/ 
~reinhold/diceware.html. 


Now, rolling a bunch of die to select 

a password is rather cumbersome 

(I’m not a table-top RPG geek), so | 
developed a script that will select any 
number of words (by default, five) and 
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select a pseudo-random list of words 


from the Diceware word list (see the esac 

article linked above for the list): done 

ppgen.sh: if [ -z $WORDLIST ]; then 

#!/bin/bash # Default word list comes from Diceware (see above link) 


WORDLIST="/usr/share/diceware/diceware.wordlist.asc" 
# Copyright ~© 2012, Trey Blancher fi 


# Licensed under a Creative Commons Attribution-ShareAlike 


# 3.0 Unported license. if [ -z $NUM ]; then 
# NUM=5 
# Inspired by XKCD fi 


# http://xkcd.com/936/ 
# #echo "$0 -f $WORDLIST -n $NUM" >&2 


# adapted from Diceware's Passphrase table-top generator 


# http://world.std.com/~reinhold/diceware. html # Algorithm uses six-sided die 
D=6 
while getopts "f:n:" opt; do 
case $opt in # Initialize RANDOM variable 
f) # Process file argument RANDOM=$RANDOM 


#echo "File argument passed: $OPTARG" >&2 
WORDLIST=$0PTARG for ((i=1; 1<=$NUM; i++) ); 
do 


for d in {1..5} # five digits per word in master list 


n) # Process number argument do 
#echo "Number argument passed: $0PTARG" >&2 WORD+=$(($(($RANDOM % $D))+1)) 
NUM=$0PTARG done 


grep $WORD $WORDLIST | awk '{print $2}' | awk '{ printf "%s ", 


$0 
\?) echo "Invalid option: -$OPTARG" >8&2 } 
echo "usage: $0 [-f <word list>] [-n <number WORD= 
of words in passphrase>]" >&2 done 
exit 1 echo 
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This script lets you choose how many 
words you want in your generated 
password and uses some tips on 
shell programming from Dave Taylor. 
There probably are better ways to 
do what I’ve done here, but this 
works for me. | use it any time | 
need a password | can remember 
(without writing it down or storing 
it in a password manager). 

—Trey 


Shawn Powers replies: | love 
Randall Monroe. When | saw his 
comic (days after writing my column, 
of course), it made me think how 
silly passwords are in general. Your 
script sounds like a good idea to me. 
Thanks for sharing. 


Platform-Agnostic Backup 

| just read the January 2013 issue 
of LJ and noticed the letter about a 
backup system for Linux. Crashplan 
looks interesting. | haven't tried it 
yet, but | plan to play around with 
its free offerings at a minimum. The 
trial looks like a good way to test 
the full experience too. | run both 
Mac and Linux computers, and | like 
that all major OSes are supported. 
—Nathan 
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Shawn Powers replies: | happen 
to agree with you. In fact, | wrote 

a piece about it in this issue’s 
Upfront section. | use it everywhere, 
and | really appreciate the generous 
free-feature offering. 


Another Vote for Crashplan 

It works on Solaris, Linux, Mac and 
Windows. | use it on Solaris and 
Windows machines, and it has been 
completely set-and-forget for me 
for a number of years now. | do the 
occasional restore to make sure that 
it is working as expected. 


Oh, Crashplan also has an Android 
application that is restore-only. It’s 
useful for getting files onto phones 
and tablets. 

—Gary Schmidt 


Shawn Powers replies: As with my 
response to Nathan, | completely 
agree. Crashplan is awesome. 


Mint+Google? 

Has Linux Mint sold its soul to Google? 
See http://forum.linuxmint.com/ 
viewtopic.php?f=157&t=108859. 
—B.r. Mintman 


| can’t speak to Linux Mint’s 
procedures and goals. | can only give 
my take on using Mint in the past: 

| thought it was very user-friendly 
and configured well out of the box, 
but | didn’t care for it much myself. 

| don’t think I’m the intended target, 
however, so that’s okay. 


| know Linux Mint always has had a 
custom Google search in the Firefox 
browser as a way to raise some 
capital. It’s possible there is more 
Google interaction, but | honestly 
don’t know. The great thing about 
Linux is that if you don’t like the 
philosophy and/or actions of one 
company, there are many other 
distros from which to choose!—Ed. 


Elliptic Curve and Putty 
Regarding the “Elliptic Curve 
Cryptography” article in the January 
2013 issue: Putty unfortunately 
doesn't support ecc/ecdsa keys, 

but it’s on the wish list. 

—Wes 


Could You Review Slax Linux? 
Could you please take the time to 
review Slax Linux? | have never seen 
this version of Linux reviewed by your 
magazine in the past or present and 
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would love to see what you think of it. 
Here is the URL: http://www.slax.org. 
—Christina Cross 


I’ve covered Slax in a video review 
before. If there are cool new 
features or significant changes, 
we might take a /ook at it. | do 
recall liking the look of Slax, 

and | appreciated its underlying 
Slackware roots.—Ed. 
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be required. The System on Module approach provides the flexibility of a fully 
customized product at a greatly reduced cost. Quantity 1 price begins at $180. 


http://www.emacinc.com/som/som9x25.htm 
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LETTERS 


My wife bought me a Rasberry Pi for my 
birthday, and | thought you might get a kick 
out of this picture of my son holding it on 

its first boot. It’s connected to the flatscreen, 
with top running in the login shell. 

—Michael Soulier 


First Boot of a Pi 


WRITE LJ A LETTER We love hearing from our readers. Please send us 
your comments and feedback via http://www.linuxjournal.com/contact. 


to ljeditor@linuxjournal.com! 
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1&1 DEDICATED - 


ERVERS 


1&1 is celebrating its 25" anniversary. 
Over the past 25 years 1&1 has grown to become 
one of the world’s leading web hosts. Today, with 
12 million customer contracts and 5000 employees 
1&1 provides superior web hosting and server 
solutions to support your business. In celebration 


here is a gift from us to you. 


Our data centers offer top security, 
Cisco firewall protection and maximum uptime. 


Y Unlimited traffic with no extra cost 


Y Parallels Plesk® Panel 11 for unlimited domains 


‘xine AOE eeaeeeeeeee OFFER ENDS 
02/28/13 


Y Mobile Server Monitoring for Android/iOS devices 


ree || Parallels 


Plesk Panel 


1and1.com 


* Offer valid for a limited time only. First year save $360 off regular price of Dedicated Server XL 6. Other terms and conditions may apply. Visit www.tand1.com for full promotional 
offer details. Program and pricing specifications and availability subject to change without notice. 1&1 and the 1&1 logo are trademarks of 1&1 Internet, all other trademarks are the 
property of their respective owners. © 2013 1&1 Internet. All rights reserved. 


Intel, the Intel logo, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation in the U.S. and/or other countries. 
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diff -u 


WHAT’S NEW IN KERNEL DEVELOPMENT 


Sasha Levin wrote a new hash 
table implementation for the kernel. 
Typically in a big, decentralized 
software project, developers aren't 
necessarily aware of the cool stuff 
being implemented in the various 
nooks and crannies of the code, 
and there’s a tendency for lots of 
alternative implementations to crop 
up everywhere. Hash tables are a 
good candidate for that. Sasha's 
patch was intended to provide a 
generic hash table that all kernel 
developers can use. 

Something like this can never find 
users unless it’s really fast. Sasha's 
implementation uses a variety of 
speed-enhancing optimizations, 
including doing as much as possible 
at compile time via macros. But, 
macros have their own little syntactic 
idiosyncrasies that can require a 
little extra care during use. Mathieu 
Desnoyers and Tejun Heo pointed 
out some improvements and bugs 
to Sasha’s code, addressing some of 
those problems. 

Darrick J. Wong took the trouble 
to create a nearly three-hour 
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gource animation of Linux kernel 
development history, going all 
the way back to the beginning: 
http://www.youtube.com/ 
watch?v=pOSqctHH9Vvy. 

It's lovely to watch. It’s also 
interesting because gource relies on 
having a computer-readable log of a 
project’s development history, while 
the early history of Linux is more 
or less a mystery. Linus Torvalds 
accepted vast numbers of patches in 
the 1990s, but he just used the UNIX 
diff and patch commands to apply 
them. The only real records of the 
patches were e-mail logs and Usenet 
logs, but these don’t give clear 
indications of which patches actually 
were accepted into the kernel, and 
when. It was only with the arrival 
of BitKeeper, and later of git, that 
a detailed record of Linux’s history 
began to be compiled. 

There have been some efforts to 
re-create some of that early history 
into its own git tree, but this is 
nowhere near as detailed as the kind 
of historical records being created 
these days. The beginning of Darrick’s 


video, therefore, just shows Linus 
doing everything for a while and then 
being joined by other developers, 
presumably once the historical record 
became more accurate. 

After six and a half years of 
maintaining the KVM code (the 
Kernel Virtual Machine), Avi 
Kivity has stepped down. His 
co-maintainer Marcelo Tosatti, 
formerly also the Linux 2.4 
maintainer, was left briefly as the 
sole KVM maintainer, until Gleb 
Natapov came in to replace Avi. 
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Everyone should use KVM. It’s an 
easy, secure way to boot up a new 
computer without having to go buy 
one. You can play around with the 
/etc directory and other strange 
internals in ways you might have 
been afraid to before. You can try 
out different Linux distributions, or 
maybe create your own, and you also 
can experiment with creating clusters 
of many interoperating systems. If 
anything goes wrong, just turn off the 
VM and start fresh. It’s very cool. 
—ZACK BROWN 


Powerful: Rhino 


Rhino M4700/M6700 

© Dell Precision M4700/M6700 
w/ Core i7 Quad (8 core) 

© 15.6"-17.3" FHD LED 
w/ X@1920x1080 

e NVidia Quadro KSO0OM 

¢ 750 GB -117B hard drive 

© Up to 32 GB RAM (1866 MHz) 

e DVD+RW or Blu-ray 

© 802.11a/b/g/n 

¢ Starts at $1190 

© E6230, E6330, E6430, E6530 
also available 


¢ High performance NVidia 3-D on an FHD RGB/LED 

e High performance Core i7 Quad CPUs, 32 GB RAM 

e Ultimate configurability — choose your laptop's features 
© One year Linux tech support — phone and email 

¢ Three year manufacturer's on-site warranty 

¢ Choice of pre-installed Linux distribution: 


EmperorLinux 


...where Linux & laptops converge 


Tablet: Raven 


Raven X230/X230 Tablet 

¢ ThinkPad X230/X230 tablet by Lenovo 
¢ 12.1" HD LED w/ X@1366x768 

© 2.6-2.9 GHz Core i7 

e Up to 16 GB RAM 

© 750 GB hard drive / 180 GB SSD 

e Pen/finger input to screen, rotation 

° Starts at $2050 

e T430, T530, W530 also available 


Rugged: Tarantula 


Tarantula CF-31 

e Panasonic Toughbook CF-31 

e Fully rugged MIL-SPEC-810G tested: 
drops, dust, moisture & more 

© 13.1" XGA TouchScreen 

e 2.4-2.53 GHz Core i5 

° Up to 8 GB RAM 

© 320-750 GB hard drive / 512 GB SSD 

© CF-19, CF-52, CF-H2 also available 


é 


www.EmperorLinux.com 
1-888-651-6686 


Model specifications and availability may vary. 
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Android Candy: Pocket 


Sf * ag sA 


#5: OOLONG 


000'-long or WOO'-long 


What Is It? 

Oolong is more of a type of tea then a single brew. It is made using an intense drying 
and oxidization process, then the leaves are curled and twisted, which gives many 
oolong teas a unique appearance. The tea can either be wrapped into small beads with 
a "tail", or more traditionally, into long, curled leaves. This tea really can't be labelled as 
"black", "white" or "green" tea, it's really somewhere in between them all, making oolong 
its own category. 


As stated, this is more of a type of tea, with a variety of tastes. It can be light and sweet, 
with flavourings like honey and fruits, or it can be heavy and have a roasted or woody 
taste. Usually the amount of oxidization and fermentation in oolong is what makes it 
"heavier". In addition to this, oolong is another tea that is popular with flavourings like 
citrus, vanilla, fruits, and others. There are also other variations, such as the creamy 
Quangzhou oolong. 


Most people are familiar 
with Instapaper and Read It 
Later. Those types of services 
are great for tagging Web 
articles for later reading, 
and in the case of Read It 
Later (now called “Pocket”), 
they do a wonderful job of 
copying articles off-line for 
reading when the Internet 
isn’t available. 

Using Pocket’s Chrome 
extension, as | browse the 
Web during the day and 
find articles I'd like to read 
later, | add them to my 
Pocket queue with a single 
click. This works perfectly 
for me, because although | 
find lots of things to read 
during the work day, | don’t 
have time to read them. 
Now, | simply can open the 
Pocket app on my Nexus 7 
and catch up on my reading 
as if it were an e-book. As 
you can see, articles are 
formatted for easy reading, 


and because they sync off-line, reading doesn’t require Internet access. 
Pocket is a great system, a really nice Android app and provides an effective 
way of staying out of trouble at work! Check it out at the Google Play store, 


or visit the Web site: http://getpocket.com. 
—SHAWN POWERS 
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Big Data 
Innovation Summit 


San Francisco 
lle TEA aU 


Join 1000+ of the industry's top minds at the world’s largest executive led 


Big Data summit. 


Explore big data technology across 6 functional and technical tracks: 


- Big Data Innovation 
- Big Data in Healthcare 
- Big Data in Finance 


Attendee Registration 
Robert Shanley 

+1 415 992 7605 
rshanley@theiegroup.com 


- Big Data in Government 
- Women in Big Data & Tech 
- Hadoop Innovation 


analytics.theiegroup.com/bigdata-sanfrancisco 
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The State of Drupal 


We asked some Drupal community 
members and leaders from all over 
the world about the current state 

of Drupal and what they are most 
looking forward to in Drupal 8. 

We got a variety of opinions from 
everyone from core developers to end 
users. Here’s what they had to say. 

Ryan Szrama: The current 
state of Drupal is frenzied. 
Everyone has a lot of 
irons in the fire (as 
always), and core devs 
are working madly 
to get awesome 
new features into 
Drupal 8. | think 
everyone feels not 
just the challenge 
but the opportunity to 
make Drupal better, the 
ecosystem larger and their 
businesses more successful. 

I'm most excited to have fago’s 
Entity API| updated and included in 
Drupal core. A robust entity system 
will make building and maintaining 
contributed modules like Drupal 
Commerce much easier than it’s 
been on Drupal 7, and we have 
both the will and code in place 
now to make this shine in Drupal 8. 
Great work all around. 


24 / MARCH 2013 / WWW.LINUXJOURNAL.COM 


Pol Dell’Aiera: Drupal 8 will be 
the bomb. 

Bert Boerland: Web services/WSCC| 
is by far the best thing happening 
to Drupal and even the Web. Web 
services first! 

Larry Garfield: Current state: the 
sands are still very shifty. Drupal 8 
still has a strong potential to be 
revolutionary and kick so 
much ass it’s not even 
funny. The trick is if we 
can close the deal and 

not leave ourselves in 

a fugly transitional 

state. That is still a 

real risk that keeps 

me up at night. 

Most excited 
about? There's 
too much to limit 

myself. Mostly | think 
the formalization of APIs that’s 
happening in many places. The 
detangling of hook_menu, EntityNG, 
the conversion of more and more 
systems to cleanly injected (and 
therefore swappable) systems, etc. 
The architectural quality of Drupal 
8 is steadily increasing, and once 
we wrap our heads around that 
transition it should be so much 
more powerful. 


Look at the distinction between 
a Content Management System, a 
Content Management Framework 
and a Web Publishing Tool. Really, 
really think about it. Then listen to 
Karen McGrane talk. 

Drupal is so close to something so 
awesome and | don’t think we even 
realize it. Stay tuned. 

Diana Montalion Dupuis: In 
general, | am concerned about the 
schism between the needs/desires 
of the development community and 
the needs/desires of people who pay 
us to solve their business problems 
with Drupal. I’m excited about many 
aspects, and as always, beyond 
grateful to the dedicated geniuses 
of the community. | also fear that 
we are doing something (from 
the “What problem do | solve?” 
perspective) different enough from 
previous versions/visions that we are 
taking a sharp left turn and actually 
making something else. 

| feel the enthusiasm, Larry's 
especially. | agree that transition 
management is a make or break 
challenge. A steady stream of 
information that connects our 
current marketshares’ investment 
in Drupal, and the Drupal shops’ 
investment in team skills, with 
the changes coming will ease that 
transition. “Drupal will now be 
widely applicable to XYZ and thus, 


extend our Web presence into ABC 
functional areas. So, train your 
team!” (In layman's terms.) 

Karoly Negyesi: Disclaimer: | 
am paid to work on some aspects 
of Drupal 8 core. Regardless, this 
my opinion. 

Current state: the core 
developers are often screaming. 
Sometimes from joy, sometimes 
from frustration. 

The new APIs are superbly 
powerful. At the same time, it'll 
take some effort to learn them. 
Parallel to this, there are concerns 
about the developer experience, 
but they were raised before the 
code freeze even—this is good. 
Previously, similar concerns only 
occurred much later in the cycle, 
and so they couldn't be addressed. 
No matter how much will we 
enhance the DX of the new APIs, 
the “surface” between the old and 
new APIs will be rough. See Larry's 
worries above about a transitional 
state. | consider that inevitable— 
just the degree Is the question. 

| love what is happening to the 
Ul. However, | always did. | am just 
a PHP guy. 

Jason Smith: |'m very interested 
in seeing where the services/WSCCI 
and configuration management 
stuff goes. Those are killer features. 

I'm a little concerned about the 
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barrier to entry being raised high 
enough due to architecture to 
discourage new folks. Drupal’s pulling 
toward enterprise, which is a good 
thing. But, my concern is that it may 
neglect that roadmap to mastery that 
makes Drupal a fantastic platform for 
open-source devs. 

Barrett Smith: I’m most excited 
for the configuration management 
changes slated for D8. Improving 
the way settings are managed will 
go a long way toward securing 
Drupal’s place in enterprise-class, 
multi-tier environments. 

Bronius Motekaitis: | love that 
the Drupal community has grown 
so large, that Drupal-powered 
projects keep increasing in size and 
complexity, and that core and the 
community are working diligently 
to keep up. | also like that user 
experience and accessibility continue 
to receive a lot of attention. 

While this is very exciting, it is 
disconcerting, because sweeping 
changes necessitate climbing up 
that steep, personal learning curve. 
By Drupal 8, | believe we in the 
community will have gotten much 
better about documenting core and 
contrib, migrations and upgrades, 
and build recipes. 

As code developers and 
application builders already 
marinated in Drupal, we love the 
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increases in power and flexibility, 
but it is important to retain the 
community of “simple, one-off 
builders” and continue to pique 
the interest of prospects: spending 
time on documentation, UX/design 
and appearance are all important in 
these regards. 

David Stagg: The biggest 
issue with Drupal 7 (and as a 
consequence, Drupal 8) is that it’s 
coming too quickly. While the core 
is being developed at such a rapid 
pace (that’s good!), a lot of modules 
and other libraries that worked 
exceptionally well in Drupal 6 aren't 
being ported over quickly enough to 
Drupal 7 or 8 (that’s bad!). 

Events are a perfect example. 
Drupal 6 has a robust set of 
modules and features that allows us 
to make an intensive event-based 
site; in Drupal 7, almost none of 
these are available. For those of us 
intently focused on the front end 
without the capabilities to create 
the modules ourselves, this can 
be tough. The best part of open 
source is the community, but when 
the development moves too fast, 
the dust that settles oftentimes 
forfeits a complete package. And 
after all, Drupal isn’t complete 
without the community. 

Donna Benjamin: More than one 
core developer has admitted they’ve 


not actually built sites with D7. 
Our commitment to the future 
means we abandon the past, and 
sometimes it seems the upgrade 
path is an afterthought. 

But I’m optimistic about the 
future—I think D8 has extraordinary 
potential. The work that’s been 
done on HTMLS5, Multilingual, 
Mobile, Config Management, Web 
Services and Authoring Experience 
is really quite phenomenal. It will be 
a game-changer. 

| just hope it’s still the kind of 
system that provides the kinds 
of solutions my clients and 


communities require. But | can’t 
deny | am worried its enterprise 
target puts it out of reach for small 
sites, for small orgs with small 
budgets, and volunteer teams. 
Many people learn Drupal in those 
environments. At the moment, we 
don’t have an alternative formal 
training and certification pathway 
to replace that pipeline. 

As for the community itself, keep 
an eye on AsiaPac: India, China, 
South East Asia and Australasia. 
We're just not on the radar for most 
EuroMericans. 

—KATHERINE DRUCKMAN 
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Design Your Own Rocket 


A lot of the software packages 
I've covered in recent articles have 
been focused strictly on doing 
computations on your machine, 
separate from the real world. 
So in this article, | explore how 
to use your computer to design 
something you can build and use 
in the real world: your own model 
rocket. Let’s take a look at the 
OpenRocket utility and see how 
it can help you design your own 
rockets. OpenRocket even can run 
simulations on your designs to show 
how they should behave in flight. 
Most distributions should include 
a package for OpenRocket. For 
example, in Ubuntu, you would 
install it with apt-get install 
openrocket. It is actually a Java 
program, so you always can download 
the jar file directly from the Web site 
(http://openrocket.sourceforge.net). 
To run it, you need to have a reasonably 
up-to-date Java VM installed as well. 
When you first start it up, you are 
presented with an empty screen, 
ready to begin designing your first 
rocket. A project window pops up, 
allowing you to enter details like 
the design name, your name and 
design notes. You can build your 
rocket from a series of components. 
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You probably will want to start with 
the nose cone by clicking on it from 
the “Add new component” window. 
It then will appear in the bottom 
section, beginning your design. 

You will notice that OpenRocket 
already begins to make calculations 
based on your design. You will see 
a small blue circle that denotes the 
center of gravity of your rocket. 
The center of gravity is the point 
through which all of the mass acts. 
There also is a small red dot that 
marks the center of pressure. This is 
the point through which all of the 
atmospheric forces act. OpenRocket 
calculates these values as you make 
changes to your design. 

You can edit almost all of the 
parameters for each component. 
There are two ways to edit these 
component parameters. You can 
double-click on the component of 
interest in the design window to 
access the edit window. There also 
is a list of the component layers in 
the top half. You can highlight the 
component of interest here and then 
click on the Edit button. 

What you can change will depend 
on which component you are trying 
to edit. The first component to 
design probably is the nose cone. 


You can select what kind of profile 
your nose cone should have, such as 
conical or ellipsoid. You can set the 
length and base diameter, as well as 
the wall thickness. But, OpenRocket 
goes even further. You can select 
the kind of material from which 
your nose cone should be made. The 
different types of materials have 
different densities, which changes 
the weight of your component. 


Rocket configuration 
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Several different material presets 
are available, but you also can 
go ahead and define your own 
custom material type. This custom 
material either can be used just 
for one design, or you can add 
it permanently to your materials 
database if it is something you will 
be using over and over again. You 
even can set the type of finish your 
rocket will have. This finish can be 
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When first starting OpenRocket, you get a new project dialog where you can enter 


details of your rocket. 
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Nose cone configuration 
File Edit Analyze Help 
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Rocket design | Flight simulations 
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Nose cone configuration 
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You can edit many details of the different components by double-clicking the 


component of interest. 


different for each component, or 
you can apply a common finish to 
your entire rocket. 

The next part of your rocket is the 
actual body tube. If your rocket is 
going to have a single stage, you 
will need only a single body tube. 
For multiple stages, you will need a 
separate body tube for each stage. 
Opening the edit window for the 
body tube allows you to change 
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the tube length, diameter and wall 
thickness. You also can change the 
material from which the body tube 
is made. 

Because body tubes usually 
contain a rocket motor, you can set 
the motor that you want to use as 
well. To do so, open the edit window 
and click on the motor tab. You 
can click the Select Motor button 
and choose from a large selection 


| UPFRONT ; 


Body tube configuration 
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The rocket engine database has lots of information on many different brands. 


of commercially available motors. motors can use OpenRocket to 
This list of motors includes technical design their rockets. 
information, such as the thrust The component list includes other 
curve and the run times, as well items like parachutes, shock cords, 
as physical characteristics like the connectors and blocks, so you 
length, diameter and weight. can design your rocket as a 

As with the other sections of the complete model. 
application, you can define your The final stage is to add tail 
own custom entries. This means fins to your rocket. As with the 
even full-on DIY model-rocket other components, there are well 
enthusiasts who build their own designed presets available that will 
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satisfy most design needs. You can 
edit some parameters to customize 
the fins to some degree. If that’s 
not enough, there is a freehand 
option where you can design your 
fins from scratch. 

OpenRocket Is not only a design 
program. You also can take your 
model rocket and run an analysis 
on it to see how it will behave in 
flight. The analysis section looks at 
individual components and shows 


Component analysis 


how they affect the stability, drag 
and roll characteristics of your 
rocket. You can set parameters 

like wind direction, angle of attack 
and speed, and calculate how it 
will behave in flight. This is great 
functionality, but OpenRocket 

goes even further. It can take your 
Original design and try to optimize 
it for the best flight characteristics. 
You can optimize based on altitude, 
velocity or some other combination 


Ja Ai 
Rocket design | Flight simulations 


Rocket 

e}Sustainer Move up 
[=D Nose cone 

Sammy Body tube 


—D 


Add new component 


Body components and fin sets 


Lid =) ca ay aa = 


L Trapezoidal Fin set 


al 
_ 
Sw 
: Elliptical Freeform Launch lug 
Component analysis 
_— : 
te] Wind direct... 0° Warnings: 
No warnings 
Angle of att... 0 + = 
‘ vex It Engine 
> 9 Mach number: 0.30 i ined 
Roll rate: Or/s - -}—_—————_ 
s 
Active stages: “Stage Motor configuration: |[E9-P] v 
: ; | _——T 
“ Stability Drag characteristics Roll dynamics 
16 Sideview Back view g Motor configuration: | [E9-P] v 
, Component CcG/cm Mass/g cP /cm Ove 
Gilet oe oo Nose cone 9.53 20.8 7.04 2 28 2% «6500631 (82S. CYB 
! ~ * satlinnfiufir tied susteneabinatinn 7 — - — Blncloatusdadh aticbiclaalioafaal nafs basdaalassalio nalaccalessebuole satiny 
__| a Rocket Body tube 25 44 25 [0 Stability-0.69 cal 
=, “= Length ah 5 ery, Tad Trapezoidal fin set 33.8 2.75 [32.5 2.8 2 CG:25.3 cm 
=i ag ase witt:motors a oral 25.3 129 21.9 4.8 | siete Pere 
~O) a 
al 3 
= 43 
LN 3a Referencelength; Scm Reference area; 19,6 cm? 
a3 
05 
3 
5 3 
ee 
= Apogee: 25.3m 
8_= Max. velocity: 47.3 m/s (Mach 0.14) 
7 = Max. acceleration: 237 m/s? 
Clickto select Shift+clickto select other Double-click to edit Click+drag to move 


You can run an analysis on the different components of your rocket. 
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of characteristics. code to the simulation to add 

Once you have a final design, extra effects. One way to do this 
you can run it through simulations. is using Python and jPype to use 
The simulator can apply different Python code in the simulations. 
conditions on your rocket, like Additionally, you have the ability to 
applying crosswinds or taking write and use your own expressions 
Coriolis effects into account, and it in the simulator. This allows you 


shows how your model rocket should to customize the simulator to a 

behave. OpenRocket uses JFreeChart great degree without having to 

to plot your rocket’s behavior based add external code. Then, you can 

on the simulation results. see the limits of what your model 
You even can add your own should be able to handle and how 


Rocket optimization 
py 


Rocket optimization 


Parameters to optimize: Available parameters: 
ESustainer 


Parameter Current Minimum Maximum 


2 CG: 36.7 cm 
® CP:33.6 cm 
at M=0.30 


® 
= 
mal Nose cone: Length 30cm 75cm [30cm 
— —Shape parameter CL) A — 
bY, r Thickness 
‘ Elliptical Freeform Launch lug 
<= } Optimization parameters 
Length 
Remove all +-Motor ignition delay 
Motor overhang es 
w —Thickness ae 
SRE ngIne: 
if 9] han apezoida fin sec block 
—Cant angle 
a Liatebt 
s 
Optimization options Required stability 
; Optimize simulation: Basic simulation [E9-P] v # Minimum stability: 4.0 a) cal 
16 Motor configuration: | [E9-P] v 
| a Optimized value: Total flight time 
Maximum stability: 40 as 50 
‘/ Peds) cet cea son Ua Vee el ved waar svar a ed cd mero Ler Seo 
Optimization goal: | Maximize value v Stability:-0.62 cal 
_ — 
= 
_— 


0 5 10 15 20 25 30 35 a0 as 50 
feed CaS M SS OOS oO MD Oe Os eS ED Se OS Best value: N/A 
| Stepcount: 11 
5. Evaluations: 14 
| Step size: 0.31% 
a | =a Start optimization 
5 Plot path 


Save path 


Apply optimization Reset Close 


“]) Apogee: 35.8m 
— Max. velocity: 51 m/s (Mach 0.15) 
- Max. acceleration: 257 m/s? 


Clickto select Shift+clickto select other Double-clickto edit Click+ drag to move 


You can optimize your rocket to maximize certain parameters. 
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Edit simulation 


rs 
Rocket design Flight simulations 


New simulation Edit simulation Run simulations Delete simulations Plot / export 


Name Motors 
Simulation 1 [E9-P]} 


Velocity off... Apogee Velocity at... | Max, velocity Max. accel... Timetoap... |Flighttime  Groundhit... 


Edit simulation 


Simulationname; Simulation 1 


a= a | 
Launch conditions Simulation options | Plot data | Export data 


aVnagninne 


Motor configuration: [E9-P] v 
Wind Launch site 
Average windspeed: 2.0|%| m/s -|_-— Latitude: 45.0|°)°N ——Ff 
jation: 2) 4] —k ngi 6 ols) ° —[- ; , 
Sideview Back view Starwars Geviseen 0.2} | (m/s Pongituge 0.0) Motor configuration: | [E9-P] v 
o* fen e Turbulence intensity; | 10.0!° % Medium Altitude: 0.0) Se | ——— ra Fg 50 
a XS sO Pak HDR SER Os ESS da tiat a Fol sah ih 17 Sa ES RA rae on ATs ew Sec NN SEL Serie BP Th ed 
| Rocket Stability:-0.62 cal 
= Length 52.5 cm, may @ CG:36.7 cm 
-ag) Mass with motors 1 Atmospheric conditions Launch rod . pa cm 
4 # Use International Standard Atmosphere Lenath: 400,0 I cm —f-_ 
om J = Angle: 0.0|2} | -— 
yr" al i Direction: oo} se | ——i— / 
> Run simulation Close 
; i 
51 
107 
. Apogee: 35.8m 
— Max. velocity: 51 m/s (Mach 0.15) 
§ 7 + Max. acceleration: 257 m/s? 
Clickto select Shift+clickto select other Double-click to edit Click+drag to move 


You can run full simulations to see how your rocket will behave in different conditions. 


high it will go under different 
conditions. This is really great 
in helping you decide when the 
weather is going to be too rough 
for your design. 

| have covered only the features 
available in the most minimal way. 
If you are into building and flying 
model rockets, your time definitely 
will be well spent poking around all 
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of OpenRocket'’s available features. 
The Rocketry Forum hosts a forum 
where you can ask for further 

help. And once you have gained 
some experience, you can give 

back by helping other new rocket 
enthusiasts. Go ahead and design 
your fleet, and get yourself out into 
the wild frontier of space! 

—JOEY BERNARD 
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Shell Game 


Many of the cool things in Linux Journal require the use of the command line. 
For us Linux users, that’s generally not a big deal, because we have a terminal 
window readily available. Some of the time, however, it’s helpful to have a shell 
account on an Internet host somewhere. 

If your Web-hosting service provides shell access, you might be able to use 
it for rudimentary command-line procedures. (In fact, Dreamhost in particular 
allows SSH tunneling through its servers for clients.) If you want to use particular 
programs like screen or irssi though, it will require something a little more robust. 

Some free shell services are available (like http://www.geekshells.org), 
but they often are very restrictive, and it can be challenging to get an account 
with them. Thankfully, if you don’t mind spending a few dollars a month, shell 
accounts are fairly common and relatively inexpensive. The Eggdrop folks have 
compiled a great list here: http://www.egghelp.org/shells.htm. 
: | Of course, 
| * Terminal - spowers@kermit: ~ — + | if you want 
to have a full- 
blown server 
on the Internet, 
it’s hard to beat 
a colocated 
Raspberry 
Pi server 
like the one 
Kyle Rankin 
Soeesaurite : talked about 
last month. 
However you 
manage it, it’s 
hard to bea 
geek without 
access to a 


File Edit View Terminal Go Help 


.... Have you mooed today?".. 


ispowers@kermit ~ $ 


Figure 1. Sometimes, you just need a shell. terminal! 
—SHAWN POWERS 
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Non-Linux FOSS 


For some reason, single-site Web applications like the 
kind created with Prism seem to have fallen out of style. 
I've always been particularly fond of Chrome-based 
applications, and yet, there doesn’t seem to be a way 
to create a single-site Web application properly on OS X 
using Chrome. Thankfully, even with modern releases 
of Chrome, it's certainly possible and still very useful. 

Two different scripts are available and freely 
downloadable for creating single-site Chrome apps on 
OS X. The first is an AppleScript file created by Mait 
Vilbiks, available at http://snar.co/chromeapp1. The 
other is a Bash script that does the same identical 
thing, but it works on the command line. It’s available 
at http://snar.co/chromeapp2. (Note: both files are 
public domain and can be modified if desired.) 

If you have ever wished Google Calendar was a 
standalone application, or if you just want to have dock 
icons for your various Web activities, either of these 
scripts work wonders. Even better, if you want to delete 
the application, simply drag it to the trash. Chrome apps 
work great as standalone applications in OS X, and 
now you can try them yourself!—SHAWN POWERS 


What should the Application be called? 


My Cool Website 
Cancel | 


Figure 1. The AppleScript version walks you through the process. 
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If this is coffee, 
please bring me 
some tea; but if this 
is tea, please bring 
me some coffee. 
—Abraham Lincoln 


Almost all my 
middle-aged 

and elderly 
acquaintances, 
including me, feel 
about 25, unless 
we haven't had our 
coffee, in which 
case we feel 107. 
—Martha Beck 


Do Lipton employees 
take coffee breaks? 
—Steven Wright 


I have measured out 
my life with coffee 
spoons.—T. S. Eliot 


To me, the smell of 
fresh-made coffee is 
one of the greatest 
inventions. 

—Hugh Jackman 


I never drink coffee 
at lunch. I find it 
keeps me awake for 
the afternoon. 
—Ronald Reagan 
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Web Development Poll 


We polled our on-line readers about their Web development preferences 
and got some surprising and not-so-surprising results. Perhaps the 
biggest surprise is that Drupal just slightly edged out WordPress. 
WordPress traditionally has been the most popular of the two among 
our readers, with Joomla a close third to Drupal’s second place. This 
time, Drupal beat out WordPress by a hair with Joomla a distant third. 
Chrome’s popularity over Firefox was a surprise, but as someone who 
tests in everything but prefers Chrome, | can’t say I’m shocked. Our 
readers’ love of vim and JQuery is not a surprise, nor is the popularity 
of Python and PHP. 

Thanks, as always, for participating in our on-line polls. The full results follow. 


1) What is your favorite browser 3) What is the best CMS to 
for testing? develop for? 
mM Chrome: 56% m Drupal: 31% 
m Firefox: 37% m WordPress: 30% 
mM Other: 3% m Other: 18% 
m Internet Explorer: 2% m Joomla: 11% 
m Safari: 2% m Plone: 4% 
m DotNetNuke: 2% 
2) What is your favorite Web mM Typo3: 2% 
programming language? m Modx: 1% 
Mm PHP: 37% m Moveable Type: 1% 
m Python: 27% 
mM Java: 20% 4) What is your favorite editor? 
m Perl: 8% mM vim: 30% 
m Ruby: 8% m Other: 18% 
BM GEdit: 14% 
Mm JEdit: 11% 
mM Kate: 6% 
™ Komodo Edit: 4% 
mM TextMate: 3% 
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5) What is your preferred 
development framework? 
m Django: 30% 

Other: 26% 

Ruby on Rails: 16% 
Zend: 9% 

CakePHP: 7% 
Codelgniter: 7% 
Symfony: 5% 


6) Which JavaScript libraries 
do you use most frequently 
(multiple answers permitted)? 
JQuery: 74% 

Node.js: 20% 

Other: 18% 

Backbone.js: 13% 

Dojo: 11% 

Ext Jo. 1 1% 

MooTools: 11% 

YUI Library: 10% 


7) What is your favorite hosting 
company? 

Amazon: 33% 

Other: 28% 

GoDaddy: 11% 

Linode: 9% 

Dreamhost: 7% 

Rackspace: 7% 

1&1: 5% 
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8) Which do you prefer to use? 
m A text editor: 37% 

m An IDE: 16% 

m Both: 47% 


9) What is your preferred revision 
control system? 


M@ Git: 63% 

mM Subversion: 18% 
m CVS: 8% 

m™ Mercurial: 8% 

mM Other: 3% 


10) Do you currently practice 
continuous integration? 

m No: 59% 

m Yes: 41% 


11) Which SQL database do you 
work with most frequently? 

m@ MySQL: 67% 

PostgreSQL: 17% 

SQLite: 8% 

Other: 5% 

MariaDB: 3% 


12) Which NoSQL database do you 
work with most frequently? 

m MongoDB: 51% 

Other: 18% 

Redis: 11% 

Cassandra: 10% 

CouchDB: 10% 


—KATHERINE DRUCKMAN 
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Crashplan, the Only 


Reason I Insta 


I'm the sort of person who doesn’t 
like to install Java. | actually don’t like 
to install Flash either, but it’s still 
tough to survive browsing the Internet 
without Flash installed. There is one 
program that makes me break my own 
rules, however, and that’s Crashplan. 


EDITORS’ 
| |! J ava CHOICE 


For years, l’ve 
been singing the praises of BackupPC, 
and for servers, | still think it’s the 
best thing going. The problem with 
BackupPC, however, is in order for it to 
work reliably, your workstations need 
to be on all the time. This is especially 


CrashPlan 


@ Backup 
ts Restore 


@ CrashPlan Central 
@ Home Server 
© MC 1D V13 


© ODIE 


© PowersBook Pro 


@ hackintosh 


© shawnbuntu 


© techosx 
f@ Friend (Free!) 


nobrrusve project tracking 


__. Folder (Free!) 


A simple, powerful tool that 
encourages teams to spend 
more time working on their 
projects and less time 
tracking them. 


Get Rid Of Me! 


CRASHPLAN+ 


Upgrade today to get: 
> Adfree desktop 

> Real-time backup 

> Priority support 


Files 


| spowers 


Inbound 


@ Disable inbound backup 
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G) Another Computer (Free!) 


Change... Total: 33,074 files / 15.3GB 


@ Invite your friends to back up to you 
@ Let your friends back up to you with this backup code: NEV 
_.. Attach a backup archive 


CRASHPLAN™ @@- 


Sign up 


(Subscription required) 


Start Backup 


Start Backup 


Start Backup 


|Start Backup 


Start Backup 


Start Backup 


Start Backup 


33,074 files 15.3GB 
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difficult with laptops. 

Crashplan is an incredibly powerful 
backup utility that allows local or 
offsite backup, and the company offers 
cloud-based storage for reasonable 
rates. Normally, | wouldn’t be so 
excited by a paid service and a non- 
open-source software package, even 
if it does offer a Linux-native client. 
The folks at Code 42, however, have 
given away the ability to swap storage 
with friends as an alternative to their 
paid-cloud-based service. If you have a 
computer at work, and a computer at 
home, you can back them up over the 


Internet to each other completely free! 
As | already mentioned, | normally 
don’t like Java-based programs like 
Crashplan, but its functionality is so 
great, | don’t mind breaking my own 
rules. More than that, | give a lot of 
credit to Code 42 for not only making 
a native Linux client, but also for 
giving away incredible functionality 
for free. If you're not backing up your 
data, be sure to consider Crashplan 
at http://www.crashplan.com. Its 
price, feature set and generous 
non-paid features make it this month's 
Editors’ Choice!—SHAWN POWERS 


LINUX JOURNAL 


Android 


Download app now in 


the Android Marketplace 


www.linuxjournal.com/android 


| 
| 
: Sinatra | HTMLS | Catalyst | Drupal 7 | Moodle 
| INTRO TO THE 
} 
i 


| L THE pest way 
dl WEB DEVELOPMENT. 


¢ Web Apps.with 
patalyst andP ~ . 
~_ Me? 
1 & < 
i f 


NEW COLUMN: 
THE OPEN-SOURCE 
CLASSROOM 


For more information about advertising opportunities within Linux Journal iPhone, iPad and 
Android apps, contact John Grogan at +1-713-344-1956 x2 or ads@linuxjournal.com. 


COLUMNS 


AT THE FORGE 


Watir 


REUVEN M. 
LERNER 


Test your Web applications by controlling real 


browsers from Ruby. 


I recently was visiting a new client 
when one of the staff developers 
asked me to help debug his 
automated test suite. | asked what 
test frameworks he used, and he 
responded by saying, “Cucumber 
and Watir”. 

Now, I've long known about 
Cucumber, and | even wrote about 
it several years ago in this column. 
It’s an advanced acceptance-testing 
framework that’s especially popular 
among Rubyists who promote 
behavior-driven development 
(BDD), an offshoot of test-driven 
development (TDD), which looks at 
a system's functionality from the 
end user’s perspective. Cucumber 
allows you to describe your tests In 
plain English, which makes the test 
specifications easy to understand, 
even for nontechnical users. 

But Watir? | knew that Watir 
(Web Application Testing In Ruby, 
pronounced “water”) is a BSD- 
licensed framework that lets you 
interact with a Web browser from 
within a Ruby program, allowing 
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you to test with a real browser (like 
the macro-based, cross-platform 
Selenium test tool) but with Ruby 
commands (like the popular Capybara 
test tool). | had last looked at Watir 
more than four years ago, at which 
time it had a fatal flaw: it worked 
only on Windows systems and only 
with Internet Explorer. 

But it seems that in the past 
few years, Watir has grown 
and expanded, with a suite of 
software projects and products 
that are increasingly flexible and 
impressive in their scope, handling 
many different types of browsers, 
operating systems and conditions. 

In this article, | take a look at Watir 
and how you can use it to test your 
Web applications more easily. My 
client's test suite, implemented with 
Watir, really impressed me, and it 
showed how with a bit of cleverness, 
you can take advantage of Watir’s 
platform-independence to ensure that 
your application works on different 
browsers and operating systems with 
a minimum of hassle—and without 
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having to worry if the JavaScript 
emulation package you're using will 
reflect the version that’s in a user's 
browser accurately. 


Installing Watir 

One of the confusing things about 
Watir is that it’s the name of one 
particular open-source project, as 
well as a set of related projects that 
have the “watir” name in them. 
Each project has slightly different 
capabilities and dependencies, and 
is maintained by a different person 
or group. 

Moreover, although documentation 
for Watir certainly exists, it’s not 
nearly as centralized, organized or 
easy to follow as many other open- 
source projects that have reached this 
maturity level. Understanding how 
to use Watir frequently requires that 
you understand the general world of 
Web browsers and automatic testing 
frameworks in order to work with it, 
and that you figure out which version 
of Watir will do what you want. 

For example, as | mentioned 
previously, Watir originally worked 
only on Windows and IE. This remains 
true for the “watir” system itself, 
also known as “watir-classic”. 

You can use Watir on non-Windows 
machines by using Webdriver—part 
of Selenium, an Apache-licensed 
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Web application testing framework. 
Just in case you missed that part, let 
me repeat it: Watir’s cross-platform 
capabilities are made possible by 
Webdriver, which is developed by 
the maintainers of the “competing” 
Selenium test automation system. 

(Wait, did | say that Webdriver 
is an Open-source product? It is, 
but it’s also a draft specification 
API from the W3C, the reference 
implementation of which is, you 
guessed it, the Webdriver software 
in the Selenium project.) 

Because Selenium is widely used 
and well maintained, and because 
the Selenium team wants to have 
connections from many languages 
and to many browsers, it’s generally 
safe to assume that you can control 
just about any Web browser 
programmatically using Webdriver. 
The Watir community has taken 
advantage of this, creating the watir- 
webdriver gem—basically, connecting 
the Watir API to the Webdriver back 
end. Of course, there are exceptions; 
| recently found that watir-webdriver 
did not yet support the just-released 
Firefox 19. A quick check on Freenode 
IRC’s #watir channel confirmed that 
Webdriver doesn’t yet support the 
most-recent Firefox version. 

The bottom line is that for virtually 
any work you'll want to do on a Linux 
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system, you'll need to install the 
watir-webdriver gem: 


sudo gem install watir-webdriver -V 


Once you have installed 
watir-webdriver, you can work with 
Chrome and Firefox. Actually, that’s 
not entirely true. You'll also need 
to install chromedriver, a program 
that lets you control Chrome via 
Webdriver (see Resources for the 
download URL). 


Watir Basics 
Once you have installed Watir, things 
should be much easier and more 
exciting. Fire up an interactive Ruby 
shell, either using IRB or the more 
modern Pry. If you’re running Ruby 
1.8, you first need to load the Ruby 
gems package: 
pry(main)> require 'rubygems' 

And in all cases, you then need to 
load the watir library: 
pry(main)> require ‘watir' 

Now I’m going to create an instance 
of Watir::Browser, the class that 
represents a Web browser, which | can 


control via Ruby. | do this by passing 
the “new” constructor method a 
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parameter indicating which kind of 
browser | want. For example, | can 
create a Chrome browser: 


pry(main)> browser = Watir::Browser.new :chrome 


In my case, this takes a little 
while to execute, as the browser 
binary starts up and then opens a 
communication channel, via Webdriver 
and Watir, with my Ruby instance. 
| often like to use the “inspect” 
method on a Ruby object to examine 
it. Here's what | get when | do this 
with my browser: 


pry(main)> print browser.inspect 
#<Watir: :Browser:0x1b4e74e97c5b6930 url="about:blank" 


=»title="about:blank"> 


Not surprisingly, my browser 
instance has both a current URL 
and a title, neither of which is 
particularly exciting. So, let’s point 
the browser to somewhere that’s a 
bit more interesting: 


pry(main)> browser.goto 'http://linuxjournal.com' 


Within a few moments, | not only 
get control back at my interactive 
Ruby prompt, but the Web browser 
also has gone to the LJ home page. 
| can ask the browser for its title 
with the “title” method: 
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pry(main)> browser.title 


=> "Linux Journal | The Original Magazine of the Linux Community" 


| similarly can invoke the “url” 
method: 


pry(main)> browser.url 


=> "http://www. linuxjournal.com/" 


Notice that although | told the 
browser to go to http://linuxjournal.com, 
it was redirected to 
http://www.linuxjournal.com, and the 
browser's current URL reflects this. 

At this point, | can use the “html” 
method to get the HTML from the 
current browser window or the 
“text” method to retrieve a version 
of the current browser, stripped 
of HTML tags. | also can go to the 
previous page (with the “back” 
method) or reload the current page 
(with the “refresh” method). 

Now, let’s say that | want to 
retrieve all of the headlines from 
the LJ site. From a quick inspection 
of the site, | can see that each 
headline is in an “h2” tag. | can 
ask Watir to retrieve all of the “h2” 
tags in a collection, from which | 
then can display the first headline’s 
text. For example: 


pry(main)> browser.h2s[2].text 
=> "Kyle Rankin to Keynote SCALE 11x" 
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But why stop there? | can retrieve 
and display all of the current headlines: 


pry(main)> browser.h2s.each {|h| puts h.text} 


I'm not going to use up all of 
my column with a list of headlines, 
but you can be sure that this does 
print all of the headlines on the 
site. Actually, it does a little more 
than that. The above code prints 
all of the h2 tags anywhere in the 
site, which includes a bit more than 
that. Upon closer inspection, | don’t 
really want all of the h2 tags on this 
page, but rather all of the h2 tags 
that are within the div whose name 
is “content-area”. 

So, what | need to do is tell Watir 
to grab the “content-area” div and 
then retrieve all of the h2s contained 
within it. To grab the content-area 
div, | use the “div” method, which 
tags a hash describing the attributes 
of the div | want: 
browser.div(id: 'content-area') 

That returns the div, which is a good 
Start. But | want the h2s within the 
div, so | can just say: 


browser.div(id: '‘content-area').h2s 


Yes, the “h2s" method that | used 
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Watir provides a number of tricks that let you find 


and work with Web pages. 


before, when executed within the 
context of a div, restricts the search 
to that div, rather than the entire 
browser window. So, | can display all 
of the latest headlines with: 


browser.div(id: ‘content-area').h2s.each {|h| puts h.text} 


And sure enough, that works very 
nicely. Watir provides methods for you 
to retrieve many different types of 
elements from within the full browser 
context, or within a more restrictive 
context. Thus, you can find a single 
div with the singular “div” method or 
a number of them matching (optional) 
criteria with the plural “divs” method. 
You can retrieve the first h2, or the 
first h2 to match optional criteria, 
with the “h2” method, or all of the 
h2s that match optional criteria with 
the plural “h2s” method. The same 
is true for “span”, “h3", “po” and 
even “a” tags. The singular method 
retrieves the first item to match your 
Stated criteria, or if multiple elements 
matched, the first from that list. 

Watir provides a number of tricks 
that let you find and work with Web 
pages. For example, | can search 
for all paragraphs whose text is the 
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word “Linux”: 


pry(main)> browser.ps(text: 'Linux').count 

Not surprisingly, this returns a value 
of 0, because no paragraph consists 
solely of the word “Linux”. However, 
if | pass a regexp object rather than a 
string, I'll get back the “p” elements 
that contain the word “Linux” 
anywhere inside them: 


pry(main)> browser.ps(text: /Linux/).count 
=> 4 


As you can see, one of the ways 
in which | check that my criteria are 
working is by using the “count” 
method. If I'm working with a single 
Watir element, | also can use the 
“flash” method to highlight the 
element briefly on the screen, in the 
browser. For example: 


pry(main)> browser.h2(text: 'Help Us Feed You Pi!').flash 


Although this makes the element 
extremely obvious in the browser for 
a few seconds, you do need to be 
looking at the browser, and have it 
scrolled to the appropriate element, 
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to see it. You also can loop through a 
number of matches and flash each of 
them in turn: 


pry(main)> browser.ps(text: /Linux/).each {|p| p.flash} 


Interaction 

Of course, Watir can do much more 
than retrieve information from the 
browser. For example, let’s assume 
that | want to execute a search on the 
LJ site. Looking at the home page, | 
see a text field and a search button 
next to it. If | enter text in that text 
field and click on the search button, 

| should see some results. 

By using the “view source” feature 
in my browser, | see that the search 
field has a class of “gsc-input”. So, | 
focus on that element, which removes 
the grayed-out “hint” text: 


pry(main)> browser. input(class: 'gsc-input').focus 
Now, | type into the element: 

pry(main)> browser.input(class:'gsc-input').send_keys('Reuven') 
Finally, | submit the search form: 

pry(main)> browser.input(class: 'gsc-search-button').click 


Sure enough, the browser submits 
the form, and | get a list of ego- 
boosting results from the LJ site. 
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Notice that | used the “send_keys” 
method when typing into the form 
element. There are other ways to 
modify the text. For example, | can 
retrieve the text of the text field with 
the “text” method: 


pry(main)> browser.input(class: 'gsc-input').text 


If | use the “text_field” method, | 
also can set the value of the text in 
the search field before clicking on it: 


pry(main)> browser.text_field(class:'gsc-input').set 'Reuven' 


pry(main)> browser. input(value: 'Search').click 


One of the best things about Watir 
is that you're not working inside an 
emulation package, but rather an 
actual browser. This means that you 
can use and test things that require 
JavaScript. For example, if | go to the 
AirBNB.com home page, | see a huge 
photo behind the simple “Where do 
you want to go?” form. That photo 
changes every 30 seconds or so, but 
| can force it to move forward or 
backward by clicking on the < and 
> signs. Is there a way for me to 
move the image forward or backward 
without clicking on these buttons 
with the mouse? 

The answer is yes, but first | have 
to find the elements that will take 
the click. Opening the HTML source, 
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| found that the < and > buttons were 
defined as <i> elements inside a div 
whose class was “arrows”. | was able 
to grab that div with: 
browser.div(class: ‘arrows') 

| then wanted to get at the <i> 
elements inside of the div. Using 
the “elements” method on the div, 
| grabbed the left-arrow: 


browser.div(class: '‘arrows').elements.first 


Of course, | double-checked that | 
had grabbed the correct element with 
the “flash” method: 
browser.div(class: '‘arrows').elements.first.flash 

Once establishing that it was the 
right item, | clicked on it: 


browser.div(class: '‘arrows').elements.first.click 


Sure enough, that did it—the front 
page of AirBNB.com switched images 
right away, rather than waiting for 
the timeout to occur. In this way, you 
can test JavaScript and Ajax events, as 
they take place in an actual browser. 


Conclusions 
The good news is that Watir is a great 


tool for testing Web applications. I've 
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been using it only a short while, and 
I'm already delighted with the sorts 
of sophisticated tests | can run. The 
fact that | can test different browsers 
automatically, work against actual 
sites and know that I’m using a 
real-world environment, rather than 
one designed for testing, is a big 
help. Watir has come a long way 
from its Windows-and-lE-only roots, 
and the developers deserve a great 
deal of credit. 

That said, there are a number 
of issues with Watir, ranging from 
the difficult-to-follow installation 
instructions, to the confusingly large 
number of related (and unrelated) 
Ruby gems, to the API, which is well 
written, but poorly documented. 
It's frustrating, for example, that | 
can retrieve the contents of a text 
area using either the “input” or 
“text_field” methods, but that only 
text_field allows me to change the 
current text value. 

Integrating Watir into existing 
test facilities, such as RSpec and 
Cucumber, appears to be possible, 
but the documentation for doing 
so (like much in the Watir world) is 
incomplete, somewhat out of date 
and spread across a number of sites. 

Should you use Watir? Based on 
what I’ve seen, the answer is yes, 
although you should be prepared to 


COLUMNS 


AT THE FORGE { 


spend some time reading blog posts, procedures, and | encourage you to 
downloading a number of versions of consider doing the same on your own 
Watir and trying to understand why projects. 


things that should work don’t. Once 
you're past the initial installation and Web developer, trainer and consultant Reuven M. Lerner 
learning curve, working with Watir is is finishing his PhD in Learning Sciences at Northwestern 
pleasant and straightforward, with an University. He lives in Modi’in, Israel, with his wife and three 
API that is second to none. I’ve started children. You can read more about him at http://lerner.co.il, 
to incorporate Watir into my own test _ or contact him at reuven@lerner.co.il. 


Resources 
For the latest versions of Watir and documentation, go to http://watir.com. 


There is now a Watir book that | downloaded and read called Homebrewer’s Guide 

to Watir by Zeljko Filipin, an active Watir community member. I’m sorry to say that 
the book isn’t well written and doesn’t include very many examples or considerations 
beyond simple documentation of what Watir can do. To his credit, the author is 
contributing all proceeds from the book to the Watir project. 


A second book, Cucumber and Cheese, by Jeff Morgan, is also available for 
$14.99, but | didn’t have a chance to read or review it. Links to both books are 
at http://watir.com/book. 


There is a Watir podcast in which prominent Watir users are interviewed about their test 
techniques, available from http://watirpodcast.com and distributed via a variety of 
networks, including iTunes. It is hosted by author and community member Zeljko Filipin. 


A variety of Watir-related projects, such as watir-grid, are available for download as 
Ruby gems. You can find a large number of Watir-related Ruby gems by searching 
for “watir” at http://rubygems.org. Other gems and resources are available from 
http://watir.com, which has a large number of pointers to other sites and software. 


You can download Chromedriver from Google Code at http://code.google.com/p/ 
chromedriver. From the downloads tab, choose the driver for either Linux32 or Linux64, 
according to your computer’s needs. After downloading Chromedriver, install it in a 
directory on your PATH, such as /usr/local/bin, or in your own personal ~/bin directory. 
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Cribbage: 


DAVE TAYLOR 


Calculating 
Hand Value 


This month, Dave talks combinatorics and factorials. 
If you’re a math geek, you’ll love the discussion. Oh, 
and there’s a little bit of shell script coding too. 


The last few months, we've been 
building a complex shell script 

to play elements of the game of 
Cribbage, demonstrating a variety 
of concepts and techniques as we 
proceed. That’s all good, and last 
month, the script expanded to 
include a “shuffle” capability and 
the ability to deal out six cards, a 
typical two-player starting hand. 

Cribbage tip: in a three-player 
game, each player gets five cards, 
and the extra is dealt directly into 
the “crib”—the extra hand counted 
after play by the dealer. 

The challenge we’re going to 
tackle this month is figuring out 
the best four cards to keep of 
those initial six, based on potential 
point values of the different hands. 
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Ultimately, a fifth, “cut card”, is 
turned up that everyone shares 
(somewhat akin to the shared 
cards in Texas Hold ‘em), but smart 
Cribbage players seek to maximize 
the four they hold and celebrate 
when the fifth adds to the point 
value of the hand, rather than count 
on that cut card to make the hand. 

So what's valuable in a Cribbage 
hand? Sequential runs of three or 
more cards, pairs or better of the 
same rank, having all the cards in 
your hand match suit (and possibly 
the cut card), combinations of card 
ranks that add up to 15, and having 
the jack of the same Suit as is on 
the cut card. 

Cards of the same rank—pairs or 
better—are tricky to count because 
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the game works on combinations. 
So 5H, 5D is worth two points 
“for the pair”, but 5H, 5D, 5S 
is worth six points, because it 
combines to three pairs: 5H+5D, 
5H+5S and 5D+5S. Four of a kind? 
A sweet 12 points and a very rare 
combination indeed. 

Critical to know: aces always 
are worth 1 when counting up 
fifteens, and all face cards are 
worth 10 points. So A+4+K Is a 
fifteen, just as 9+6 is, though for 
pairs, it’s the same rank, not just 
pairs of face cards. In other words, 
J+K is not a pair. 

Where counting gets tricky is 
those combinations of fifteens. 
A great hand, for example, is 
6S, 7H, 8D, 8S, which offers two 
points for 7H+8D, two points for 
7H+8S, three points for 6S+7H+8D, 
another three points for 68S+7H+8S 
and a final two points for the pair 
8D+8S—very nice. If the cut card is 
another 7 or, even better, another 
8, you can see where there are a 
lot of points in that potential hand! 

The challenge, of course, is doing 
all of this programmatically. When 
we left off the program last month, 
it could deal six cards to us and 
sort them in increasing rank order, 
which is helpful, particularly in 
recognizing runs. 
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Calculating All Four-Card 
Combinations 

The first problem we have is actually 
earlier than anything to do with card 
values. It’s to calculate all possible 
four-card combinations out of a 
six-card hand. It’s combinatorics— 
something | really enjoyed back 
when | took that class in college for 
my CS degree. 

What we’re looking at Is actually 
factorial math. So “six choose four” is 
really 6! minus 2!, which is generally 
written as shown in Figure 1, where n 
is the total number of cards, and m is 
the subset we want to choose. In this 
case, it’s 6! / 4!(6! - 4!). 


n! 
ml(n - m)! 


Figure 1. Equation for “Six Choose Four” 


A factorial number is calculated 
as the multiple of all decreasing 
numbers, so4! =4x3x2x 1, and 
Gleaox5x4%3x2% 1. OF course, 
the “x 1” part is pointless, so we 
can skip it. 4! = 24 and 6! = 720, 
which means that we have 15 card 
combinations to check for points. 
(If we had only five cards dealt 
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because it was a three- or four-person 
Cribbage game, we'd have only five 
combinations to evaluate instead.) 

The question is, how do we figure 
out all 15 of those combinations 
easily? | mean, | could hard-code 
all 15 of them—it’s only four digits 
per subset, after all—but that seems 
less interesting than trying to solve 
the more-general mathematical 
equation. And yes, | can see myself 
getting sidetracked as | proceed, 
but that’s part of the fun of 
programming when you're not on 
a deadline, right? 

Um, never mind. Let’s just take 
as a given that for six-choose- 
four, there are these specific 
combinations and no more: 
115253;4)) 1125 3h 11d 38}, 
bee Apoyo, 1 152,5,61s 
(1,3,4, 5) 115374,61 11,3,5,61 
11,4,5,6},412,3,4, 5}, 42;3;4.6), 
{2,3,5,6}, {2,4,5,6} and {3,4,5,6}. 

If we're working with five- 
choose-four, the combinations 
are {a,b,c,d}, {a,b,c,e}, {a,b,d,e}, 
{a,c,d,e} and {b,c,d,e}. 

The easy way to code this is as an 
array of arrays. Let’s do it this way: 


# six choose four 

sixfour[0]="0 1 2 3"; 
sixfour[2]="0 1 2 5"; 
sixfour[4]="0 1 3 5"; 


sixfour[1]="0 1 2 4" 
sixfour[3]="0 1 3 4" 
sixfour[5]="0 1 4 5" 
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sixfour[6]="0 2 3 4"; sixfour[7]="0 2 3 5" 
sixfour[8]="0 2 4 5";  sixfour[9]="0 3 4 5" 
sixfour[10]="1 2 3 4"; sixfour[11]="1 2 3 5" 
sixfour[12]="1 2 4 5"; sixfour[13]="1 3 4 5" 
sixfour[14]="2 3 4 5" 


If you're paying close attention 
to the script we've been creating, 
you know that our hand is indexed 
Starting at 0, so the card values are 
O-5 instead. That's easily fixed by 
shifting every value down one in the 
predefined arrays. 

Now it’s just a matter of interpreting 
the different sixfour array values 
as a set of four cards to select, and 
here's how | do it: 


for subhand in {@. .14} 
do 
/bin/echo -n "Subhand ${subhand}:" 
for thecard in ${sixfour [$subhand] } 
do 
showcard ${hand[$thecard] } 


/bin/echo -n $showcardvalue" 


done 


echo 
done 


The echo statements (remember 
-n prevents a carriage return being 
appended to the output) aren't 
important to the final code, but for 
this interim solution, you'll see how 
it works by me just tucking it into the 
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end of the program, just after the Subhand 10: 5C 7H 8H 9D 
code that deals, sorts and shows the Subhand 11: 5C 7H 8H JS 
full six-card hand: Subhand 12: 5C 7H 9D JS 

Subhand 13: 5C 8H 9D JS 
Hand? AS, 5C, 7H, BH, 9D; JS. Subhand 14: 7H 8H 9D JS 
Subhand 0: AS 5C 7H 8H 
Subhand 1: AS 5C 7H 9D That’s a lot of four-card hands, but 
Subfiand 2? AS 5C 7H JS at least we've got them. Next month, 
Subhand 3: AS 5C 8H 9D we'll actually start writing the code to 
Subhand 4: AS 5C 8H JS evaluate hand values. Stay tuned! m 
Subhand 5: AS SC 9D JS 
Subhand 6: AS 7H 8H 9D Dave Taylor has been hacking shell scripts for more than 30 years. 
Subhand 7: AS 7H 8H JS Really. He's the author of the popular Wicked Cool Shell Scripts 
Subhand 8: AS 7H 9D JS and can be found on Twitter as @Davelaylor and more generally 
Subhand 9: AS 8H 9D JS at http://www.DaveTaylorOnline.com. 
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Temper PI 


KYLE RANKIN 


Kyle returns to his quest to control a beer fridge with ever 
cheaper and smaller Linux devices. Today’s candidate is a 


Raspberry Pi. 


It was inevitable. Back when 

the Raspberry Pi was announced, 

| knew | eventually would use one 
to power a beer fridge. If you have 
been following my column through 
the years, you know that three years 
ago (see my Hack and / column titled 
“Temper Temper” in the August 2010 
issue, http://www.linuxjournal.com/ 
article/10809), | set up a temperature 
controller for my beer fermenting 
fridge with an X10 serial controller 
to control the power to the fridge 
and a heating pad, an inexpensive 
TEMPer USB thermometer to take 
the fridge temperature, and a 
simple Perl script. 

As described in the original 
article, everything was connected 
to a spare Debian laptop | had 
lying around, and the setup worked 
great. Every minute, my Perl script 
would launch, take the temperature 
and control the power to a heating 
pad at the bottom of the fridge 
or the fridge itself, depending on 
whether it needed to be warmer 
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or cooler. A few months later (see 
my December 2010 Hack and / 
column “Working on My Temper”, 
http://www.linuxjournal.com/ 
article/10904), | decided that the 
laptop was overkill for this use case, 
so | replaced it with a low-power 
Pogoplug NAS device that was 
modified to boot Plugbox Linux, 
an Arch derivative. The Pogoplug 
has been powering my beer fridge 
reliably ever since. 

That brings us to today. | just 
happened to have a Spare insulated 
cabinet on the other side of the 
garage that | wanted to use as excess 
fermentation capacity. Unfortunately, 
the cabinet is too far away from the 
fridge for me to use the Pogoplug with 
an additional temperature probe, so | 
had to work out a different solution. 
| happened to have a spare Raspberry 
Pi lying around and realized it would 
be perfect for the job. All | needed to 
do was buy a new TEMPer USB probe 
and copy over my Perl script. The only 
big change I'd need was to have the 
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The problem with cheap electronics is that sometimes 
the internals change without your knowing. 


script ssh back in to the Pogoplug so Debian distribution. This procedure 
it could control the X10 devices (| have has been documented many times 


only one X10 serial adapter). before, so | won’t document it here. 
Because | was using a Debian-based 

Prepare the Raspberry Pi release, | figured | even could follow 

| didn’t really need anything fancy the same steps from my original 

for this setup. In fact, Arduino fans “Temper Temper” column. 

who read this probably would say 

that even the Raspberry Pi is overkill New TEMPer Thermometers 

for such a simple project. | decided to. The problem with cheap electronics is 

use the standard Raspbian “wheezy” that sometimes the internals change 
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without your knowing. Apparently, 
there are different USB thermometers 
all under the TEMPer name with the 
same packaging and overall look. 
Although I’m sure they all work with 
their included Windows software, it 
turns out that they need completely 
different software under Linux. 
Wouldn’t you know it, the second 
TEMPer probe | bought turned out to 
be a different revision, so it requires a 
completely different set of software. 

You can tell which TEMPer 
thermometer you have with dmesg 
and lsusb. If the dmesg output 
looks like this: 


input: PCsensor Temper as 
/devices/platform/orion-ehci.@/usb1/1-1/1-1.1/1-1.1:1.0/input/inputo 
generic-usb 0003:1130:660C.0001: input: USB HID v1.10 Keyboard 

»[ PCsensor Temper] 

on usb-orion-ehci.@-1.1/inputO 

usb 1-1.3: new high speed USB device using orion-ehci and address 5 
input: PCsensor Temper as 
/devices/platform/orion-ehci.@/usb1/1-1/1-1.1/1-1.1:1.1/input/input1 


generic-usb 0003:1130:660C.0002: input: USB HID v1.10 Device 


»[ PCsensor Temper] on usb-orion-ehci.0-1.1/input1 


and you see something like this in lsusb: 


$ lsusb 


Bus 001 Device 051: ID 1130:660c Tenx Technology, Inc. 


you have the older version of the TEMPer 
probe, and you can follow the steps from 
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my original “Temper Temper” column. 
If instead dmesg says this: 


3.213110] usb 1-1.3: new low-speed USB device number 
4 using dwc_otg 
3.339127] usb 1-1.3: New USB device found, 


»jidVendor=0c45, idProduct=7401 


3.355218] usb 1-1.3: New USB device strings: 
Mfr=1, Product=2, SerialNumber=0 
3.377771] usb 1-1.3: Product: TEMPerV1.2 


3.392684] usb 1-1.3: Manufacturer: RDing 


3.420037] input: RDing TEMPerV1.2 as 


»/devices/platform/bcm2708_usb/usb1/1-1/1-1.3/1-1.3:1.0/input/inputO 


3.436838] generic-usb 0003:0C45:7401.0001: input: 
USB HID v1.10 Keyboard 


RDing TEMPerV1.2] on usb-bcm2708_usb-1.3/inputO 


3.465103] generic-usb 0003:0C45:7401.0002: hiddevd: 


USB HID v1.10 Device 


RDing TEMPerV1.2] on usb-bcm2708_usb-1.3/input1 
and lsusb Says: 


$ Lsusb 
Bus 001 Device 005: ID 0c45:7401 Microdia 


then congratulations, you have the 
new TEMPer probe and will have to 
use completely different software. 


Temper.py 

The original software project to 
control these new-style TEMPer 
probes was at http://www. isp-sl.com/ 
ocsensor-1.0.0.tgz, but right before 
| started writing this article, | got 
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word from Philipp Adelt that he 
had updated that project recently 
to work with Python. The updated 
project is hosted on Github at 
https://github.com/padelt/ 
pcsensor-temper, and the Python 
version is at https://github.com/ 
padelt/temper-python and has 
additional features, such as multiple 
probe management and SNMP 
Support. So to get started with this 
project, | first needed to install 

git and then install a few Python 
libraries to provide USB support: 


$ sudo apt-get install git python-usb 


(Note that the project page also tells 
you to install the python-setuptools 
package and the snmp-passpersist 
Python library, but as I’m not planning 
to use SNMP, | skipped that step.) 

With git installed, | pulled down the 
latest release of temper-python: 


$ git clone git://github.com/padelt/temper-python. git 
Cloning into 'temper-python'... 

remote: Counting objects: 17, done. 

remote: Compressing objects: 100% (13/13), done. 
remote: Total 17 (delta 4), reused 15 (delta 2) 
Receiving objects: 100% (17/17), 19.07 KiB, done. 


Resolving deltas: 100% (4/4), done. 


The main Python program can 
be found under temper-python/ 
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src/temper.py, and the project also 
includes a sample udev rule you can 
copy to /etc/udev/rules.d if you want 
access to the TEMPer probe by a user 
other than root. | my case, | was fine 
with root-only access, so | left the 
udev rules alone. 

If you install the python-usb libraries 
correctly, and you have the supported 
TEMPer device, you should see output 
like the following: 
$ sudo ./temper-python/src/temper. py 
Found 1 devices 
Device #0: 17.6°C 63.7°F 

Once you see this, you know the 
temperature probe is working. | don’t 
like running system programs like this 
within a home directory, so | decided 
to copy it up to /usr/local/sbin: 


$ sudo cp ./temper-python/src/temper.py /usr/local/sbin/ 


Now in my case, | wanted to act 
on this temperature output, and 
| realized that my old temper.pl 
wrapper script wasn’t going to cut It. 
Although | certainly could just modify 
it a bit to work with the new output, 
| figured a Perl script that called a 
Python script was just asking for too 
much hate mail. Instead, | decided to 
write a new simple wrapper script in 
bash called /usr/local/sbin/temper: 
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#!/bin/bash 


TEMP_MIN="55" 


TEMP_MAX="65" 


LOGFILE='/var/log/temper.log' 
TIME="date +"%b %d %T"~ 


TEMPERATURE=/usr/local/sbin/temper.py 2>/dev/null | 


tail -nl | cut -f4 -d ' ' | sed 's/.F$//'~ 
if [[ $TEMPERATURE == "" ]]; then 

echo ERROR 

exit 1 
fi 


# B6 = peltier cooler B7 = heater 


if [[ $TEMPERATURE < $(( $TEMP_MIN - 1 )) ]]; then 
ssh pogoplug "/usr/local/bin/br --port /dev/ttyUSBO 
ssh pogoplug "/usr/local/bin/br --port /dev/ttyUSBO 
echo -e "$TIME\t$TEMPERATURE\tHON" >> $LOGFILE 

elif [[ $TEMPERATURE < $TEMP_MIN ]]; then 
ssh pogoplug "/usr/local/bin/br --port /dev/ttyUSBO 
ssh pogoplug "/usr/local/bin/br --port /dev/ttyUSBO 
echo -e "$TIME\t$TEMPERATURE\tOFF" >> $LOGFILE 

elif [[ $TEMPERATURE > $TEMP_MAX ]]; then 
ssh pogoplug "/usr/local/bin/br --port /dev/ttyUSBO 
ssh pogoplug "/usr/local/bin/br --port /dev/ttyUSBO 
echo -e "$TIME\t$TEMPERATURE\tCON" >> $LOGFILE 

else 


echo -e "$TIME\t$TEMPERATURE\t" >> $LOGFILE 


B7 ON" 


B6 OFF" 


B6 OFF" 


B7 OFF" 


B6 ON" 


B7 OFF" 


Although the logic of this script is 
similar to my old temper.pl, | just call 
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the temper.py script and use some 
pipes to pull out the temperature 
data | need. In this case, | also have 
to ssh back to my machine named 
“yogoplug” to run /usr/local/bin/br 
(the bottlerocket software that controls 
my X10 devices). This means | need 
to run ssh-keygen as the root user 
and then run ssh-copy-id to copy 
my public key to the pogoplug host. 
If | had wanted to replace my existing 
Pogoplug with a Raspberry Pi, | could 
just apt-get install bottlerocket, 
connect a USB-to-serial adapter with 
my X10 serial controller and run the 
br commands directly. 

The final step is set up a cronjob. 
For that, | just create a file in called 
/etc/cron.d/temper with these contents: 


* *  * * * root /usr/local/sbin/temper 2>/dev/null 


With that file in place, every 
minute, my script will probe the 
temperature and control the power to 
extra X10 appliances | have in place 
via the Pogoplug. It seems like | keep 
replacing these temperature probe 
systems with simpler and cheaper 
Linux machines. | wonder what’s next? 


Kyle Rankin is a Sr. Systems Administrator in the San Francisco 
Bay Area and the author of a number of books, including The 

Official Ubuntu Server Book, Knoppix Hacks and Ubuntu Hacks. 
He is currently the president of the North Bay Linux Users’ Group. 
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Dynamic DNS— 
an Object Lesson 


SHAWN POWERS 


in Problem Solving 


Got a computer problem? Learn to wield digital duct tape 


like a pro! 


The other day in the Linux Journal 
IRC room (#linuxjournal on Freenode), 
| was whining to the channel about 
no-ip.com deleting my account 
without warning. My home IP address 
hadn't changed in a couple months, 
and because there was no update, it 
appeared abandoned. The problem 
was, although the IP address hadn’t 
changed, | was using the Dynamic DNS 
domain name to connect to my house. 
When the account was deleted, the 
domain name wouldn't resolve, and | 


couldn‘t connect to my house anymore. 


The folks in the IRC channel 
were very helpful in recommending 
alternate Dynamic DNS hosting 
services, and although there are still 
a few free options out there, | was 
frustrated by relying on someone else 
to manage my DNS services. So, like 
any nerd, | tried to figure out a way 
to host a Dynamic DNS service on my 
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own. | thought it would be a simple 
apt-get install on my colocated 
server, but it turns out there’s not a 
simple Dynamic DNS server package 
out there—at least, not one | could 
find. So, again like any particularly 
nerdy nerd, | decided to roll my own. 
It’s not elegant, it’s not pretty, and 
it’s really just a bunch of cheap hacks. 
But, it’s a great example of solving a 
problem using existing tools, so in this 
article, | explain the process. 

First, It’s important to point out 
a few things. The purpose of this 
article is not really to explain the best 
way to make a self-hosted Dynamic 
DNS system. In fact, there probably 
are a dozen better ways to do the 
same thing. That's sort of the point. 
The more familiar you are with Linux 
tools, the more resourceful you can 
be when it comes to problem solving. 
| go through the steps | took, and 
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hopefully most readers can take my 
method and improve on it several 
times over. That type of collaboration 
is what makes the Open Source 
community so great! Let's get started. 


My Particular Toolbox 

We all have a slightly different bag 
of tricks. I’m in the “extremely 
lucky” category, thanks to Kyle 
Rankin tipping me off about the free 
Raspberry Pi colocation service. A 
full-blown Linux box with a static 

IP on the Internet is truly the sonic 
screwdriver when it comes to these 
sorts of things, but perhaps someone 
else’s solution won't require a 
complete server. 

Along with the Raspberry Pi server, 
| have a Linux server at home, a home 
router and a handful of domains | 
own. | also have accounts on several 
Web hosting servers, and accounts 
with cloud-based storage like Dropbox, 
Google Drive and a handful of others. 


First Problem—What’s My IP? 

The beauty of Dynamic DNS hosting is 
that regardless of what your home IP 
address is, the same domain name will 
resolve even if it changes. Granted, 

my home IP address hadn’t changed 

for months, but | still used the DNS 
name to access it. Therefore, when my 
account at no-ip.com was deleted, | had 
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no way to tell what my home IP was. 
There are plenty of sites on the 
Internet that will return your IP address. 
Because | just had my Dynamic DNS 
account deleted, | really didn’t want 
to depend on a free on-line service for 
detecting my IP address. Thankfully, 
this piece of the puzzle was simple. A 
few lines of PHP hosted on any Web 
host that supports PHP Is all it takes. 
For example, my IP detection script 
is hosted at http://snar.co/ip (my 
personal domain—feel free to use it). 
See it in action in Figure 1. It contains 
nothing more than this: 


<?php 
// Save the IP to a variable 
$ip_address = $ SERVER['REMOTE_ADDR']; 


// To display the IP: 
echo $ip_address; 
?> 


The frustrating part is that although 
| had a way to detect my IP address at 


<a © @ | 1 snar.cosip: 


71.225.225.35 


Figure 1. Unlike many “what is my IP” 
services, my script returns nothing but 
an IP address. 
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home, | wasn’t actually at home, so | 
had a catch-22 situation. Thankfully, 
my wife was home. | texted her and 
asked her to visit my snar.co address 
and let me know the IP address she 
got back. Once | had that IP address, 
| could connect to my home server 
and set up some automation. 


Let’s Never Do This Again 

It’s not that | don’t love texting my 
wife, it’s just that hoping someone 

is home to check an IP address Is 

not the best way to roll your own 
DNS. There's also the possibility that 
whatever Dynamic DNS solution | 
dream up might fail, and | want to 
make sure | always can figure out my 
home IP address. 

There are a couple different ways | 
considered for making my IP address 
always accessible. The simplest was to 
set up a cron job to upload regularly 
the results of my PHP script to my 
Raspberry Pi or to one of my Web 
hosts. In order to do that, I’d simply 
have to set up SSH keys so my home 
server could upload a file without 
any interactive authentication. This 
is, in fact, what | recommend. As it 
happens, however, I'm lazy. What | 
actually did was set up a cron job that 
copied my IP address into a text file 
inside my Dropbox folder. It’s not a 
better solution than scp-ing, but the 
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end result is the same. Here’s what my 
cron job looks like: 


1 * * * * /usr/bin/wget -r --quiet -0 ~/Dropbox/Public/IP.txt 


= 'http://snar.co/ip' 


It basically updates my Dropbox 
folder every hour with my current 
home IP address. Because Dropbox 
syncs onto every device and computer 
| own, it’s always readily accessible. 


But That’s Still Not DNS 
Now things get a little complicated. 
Because | have a full-blown server and 
a handful of domains to use, it would 
make sense to set up BIND and serve 
out a subdomain. BIND does have the 
ability to change a host entry with a 
remote update command. It requires 
setting up encryption keys, and of 
course, the BIND damon has to be 
configured properly. Remember when 
| said | was lazy? It’s still true. Because 
alll wanted to do was serve up a single 
domain name for my home IP address, 
| opted for something simpler. 
DNSMasq is a very simple daemon 
that runs on my Linux-based home 
routers. It handles both DHCP services 
and DNS resolution. In both cases, 
the services are very stripped down 
and simplistic, but if all you need is 
simple DNS resolution, it doesn’t get 
simpler than DNSMasq. It will look at 
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the server's /etc/hosts file and serve 
out those entries when queried. All | 
had to do was get my home IP address 
into my server's /etc/hosts file, and 
regularly send DNSMasq the HUP 
signal to reload its files. A simplistic 
DNS server was the final piece to the 
puzzle. Next came implementation. 


Putting It All Together 

The first step was to create a DNS entry 
that | could update with DNSMasq. This 
is simpler than most people realize. | 
just added an NS record pointing to my 
Linux server. So basically, | have an entry 
that looks like this: 


home.mydomain.org. IN NS server.mydomain.org. 


That means, “when resolving 
home.mydomain.org, or any subdomain 
of it, ask server.mydomain.org for the 
address.” This is exactly what | want, 
because then any time | (or anyone else) 
tries to access home.mydomain.org, 
it will ask my server to resolve the 
name. The only thing left to do is to get 
my server, which is running DNSMasq, 
to respond with the proper IP address. 
That means a couple more cron jobs. 

Remember my cool little IP.txt file 
| keep in my Dropbox? Well, in order 
to hack together the /etc/hosts file 
on my server, | had to modify my PHP 
script a little. In order to create an 
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output compatible with /etc/hosts, | 
changed it to: 


<?php 
// Save the IP to a variable 
$ip_address = $ SERVER['REMOTE_ADDR']; 


// To display the IP: 
echo $ip_address; 
echo " home.mydomain.org"; 


?> 


Note the space before “home”. Now 
the file in my Public Dropbox folder is 
a properly formatted /etc/hosts line. In 
order to combine that with my original 
hosts file, | created a folder /etc/hosts.d/ 
on my server, and copied /etc/hosts to 
/etc/hosts.d/00-original. 

Still with me? The last step is to run 
the following script on the server. | 
do this script every hour, so if my IP 
address changes, it should take at the 
most an hour before it’s corrected. 
Here is the server script: 


#!/bin/bash 
/usr/bin/wget -r --quiet -O /etc/hosts.d/home 
= 'https://d1.dropbox.com/xxx/IP.txt' 

cat /etc/hosts.d/* > /etc/hosts 


killall -SIGHUP dnsmasq 
The first line retrieves the current IP 
address stored in my Dropbox Public 


folder. The second line creates a new 
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/etc/hosts file by concatenating all 
the files in /etc/hosts.d/. Then finally, 
| send the SIGHUP signal to dnsmasaq, 
so it will reload the /etc/hosts file. 


Final Thoughts 

The thing | really like about this 
example as a way to demonstrate 
problem solving is that there are so 
many different ways to accomplish the 
same results. My solution is far from 
the best. Off the top of my head: 


m | could have the script on my home 
server check for a change in IP 
address rather than just constantly 
updating. If there was a change, 
it could start the update process 
on my remote server rather than 
updating the hosts file every hour 
whether it needs it or not. 


mM Depending on what DNS hosting 
company you use, It’s probably 
possible to change an address 
with a remote command. It’s also 
possible there are free DNS servers 
out there directly supported by a 
client like ddclient. 


m Because my solution requires a 
remote Linux server with a static 
IP, it makes my specific solution 
inaccessible to many people. That 
just means you need to think harder 
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in order to dream up a solution! 


Where to Go from Here 
It seems apropos to make my disclaimer 
again: the process | just explained is 
not the most efficient way to solve 
the problem of changing IP addresses. 
My methods are crude, my scripts are 
simplistic, and | haven't included any 
error correcting whatsoever. (What 
happens if | can’t download the file 
from Dropbox? Will my script fail? 
Probably!) The purpose of this article is 
to make you think. Linux gives us tools 
that are powerful, flexible and above 
all else, useful. Sometimes you need to 
create a little digital duct tape and solve 
a problem on the fly. 

What if you can’t seem to come 
up with a solution to your particular 
problem? That's where the Linux 
community really shines. Stop in at 
the #linuxjournal channel, or attend 
a local LUG meeting. Folks there are 
much like me and are eager to help 
solve problems. Everyone loves a 
puzzle, and when you get to solve it 
with Linux? Awesome! ™ 


Shawn Powers is the Associate Editor for Linux Journal. He's 
also the Gadget Guy for LinuxJournal.com, and he has an 
interesting collection of vintage Garfield coffee mugs. Don't let 
his silly hairdo fool you, he’s a pretty ordinary guy and can be 
reached via e-mail at shawn @linuxjournal.com. Or, swing by 
the #linuxjournal IRC channel on Freenode.net. 
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NEW PRODUCTS 


Soekris Engineering, Inc.’s | oa 
lan1841 | : 


The US-manufactured lan1841 from Soekris 
Engineering, Inc., is a short standard-profile PCI 
Express board featuring four independent Intel 
82574L Gigabit Ethernet controllers for systems 
where the highest number of ports is required. 
The lan1841 is designed to work either with the 
Soekris net6501 for communications applications 
or as a standalone PCI Express LAN card. Together 
with the Soekris’ net6501 series featuring a 1U 
19" rackmount case, the net6501 PCI-Express 
riser and two lan1841s, users can configure a reliable, low-power rackmount unit 
with 12 ports. Alternatively, those looking for a four-port PCI Express Gigabit LAN 
card at a reasonable price for use with other hardware will find their needs met as 
well. The lan1841 features a three-year warranty. 

http://soekris.com 


Open E R P OpenERP 7.0 


OPENS SOURCE BUSINESS APPLICATIONS [Me essence of what's novel in the 
OpenERP 7.0 open-source business suite 


boils down to the concept of “Integrated Apps”, which the company says “bring together 
the best of the ERP and the standalone app world”. The idea is that as customers proceed 
to install applications, the new ones will integrate automatically with existing ones. Other 
aspects of the complete product redesign include a full-featured library of open-source 
business apps and out-of-the-box use with no prior configuration, all driven using “a 
stunning user interface”. OpenERP 7.0 comes with new modules including touchscreen 
POS, contract management, fleet management apps, LinkedIn and Google docs integration, 
social collaboration between apps as an alternative to traditional e-mail, innovative kanban 


views to manage sales pipeline and projects, and an integrated customer portal, among 
others. The portal includes hundreds of modules developed by the community. OpenERP 
7.0. is available either as an on-site deployment or via OpenERP’s SaaS platform. 
http://www.openerp.com 
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NEW PRODUCTS 


Ben Simonds’ Blender Master 


JEAN ip)a°a | Class (No Starch Press) 


MASTER CLASS The miracle that is open-source has given us 


a hands-on guide to modeling, scul materials, and rendering 


stunningly powerful free software like the Blender 3-D 
graphics tool. Harness the power of Blender with the 
new book Blender Master Class: A Hands-On Guide 
to Modeling, Sculpting, Materials, and Rendering by 
3-D artist and animator Ben Simonds. Simonds’ goal 
with Master Class, which is filled with eye-grabbing 
full-color artwork, is to inspire and equip new users 
to create their own masterpieces. Simonds walks 
readers through three complete projects: a muscle- 
bound gargoyle, a futuristic robotic spider and ancient temple ruins overgrown with 
jungle. In the process, readers master the Blender interface and learn how to model and 
sculpt amazing models. Other topics include working with reference and concept art in 
Blender and GIMP, utilizing textures from GIMP, mastering the art of creating materials 
and lighting for scenes and rendering and compositing images to create striking images. 
http://www.nostarch.com 


AcousticSheep’s Bluetooth = ¢ — 


SleepPhones Wireless 


Ben Simonds 


Counting sheep to get to sleep has never been 
so passé with the release of AcousticSheep LLC's 
Bluetooth SleepPhones Wireless, the company’s latest addition to its line of doctor- 
designed miniature headphones integrated into comfortable, washable sports-style 
headbands. AcousticSheep assures users of SleepPhones that they will “fall asleep 
faster and drug-free” while using the product. Although AcousticSheep already 

offers similar wired products without hard earbuds not only for sleeping but also for 
exercising, this new product eliminates cords entirely by playing back media from any 
Bluetooth-enabled device. SleepPhones Wireless was selected as an Innovations Award 
Honoree from the Consumer Electronics Association in advance of CES 2013. 
http://www.sleepphones.com 
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Velocity Micro’s Cruz D610 
and Cruz Q610 


Hardware-maker Velocity Micro’s design philosophy is 
to provide products that are “aesthetically pleasing, 
blazingly fast, competitively priced and designed 

and built in the USA”. Although Velocity has yet to 
add “powered by Linux” to the list, the company 
recently released the Cruz D610 and Cruz Q610, 
affordable Android 4.1 tablets powered by dual-core 
and quad-core ARM processors, respectively. Both 
devices feature a 10" capacitive touchscreen, camera 
(D610 front-facing, Q610 front- and rear-facing), Google Play, Flash, custom software 
suite and access to Kindle for Android and the Amazon app store. The company says 
that these new tablets achieve this year’s goal to blend the enthusiast level and the 
mainstream to create products with mass appeal. 

http://www.velocitymicro.com 


Red Hat Enterprise Linux 
/ red Nat. The tried-and-true Red Hat Enterprise Linux 
(RHEL) open-source enterprise platform recently 
was upgraded to version 6.4. The new release, which continues in RHEL’s tradition 
of delivering technology that is tested and stable, includes updates to the platform's 
existing feature set and provides new functionality in the areas of identity management, 
filesystem, virtualization, storage and productivity tools. Identity management is 
improved via SSSD enhancements that strengthen the interoperability experience with 
Microsoft Active Directory. Filesystem enhancements from pNFS deliver performance 
improvements with the addition of Direct I/O for faster data access. Virtualization is 


more robust, thanks to the inclusion of the Microsoft Hyper-V Linux drivers and other 
improvements. Storage management is now easier thanks to new system log features 


that simplify the process of locating specific devices. Productivity-tool upgrades 
include interoperability improvements with Microsoft Exchange, calendar support in 
Evolution and support for newer Wacom tablets. 

http://www.redhat.com 
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NEW PRODUCTS 


Carlos Sessa’s Android Hacks 
(Manning) 


} News from the DIY Android desk is the availability of Carlos 

Cats Ses LAK Sessa’s Android Hacks, a new Manning Publications book of 50 
ee short and simple programming hacks that are destined to “save 
you time, stretch your skills and maybe even make you smile”. 
Peco The 50 “little gems” are organized into categories such as Layout, 


Animation, Patterns and UX, and each covers just a few pages, 
including annotated source code. Some examples of the hacks include creating a wizard form 
using a Gallery, using Scala inside Android, adding an MP3 to the Media ContentProvider and 
batching database operations. Most hacks work with Android 2.x and greater. 
http://manning.com/sessa 


JetBrains IntelliJ IDEA 


The headlining new feature in IntelliJ IDEA 12, 
the latest volume of JetBrains’ Java IDE for Web, 
enterprise and mobile development, is a brand- 
new, much faster compiler. JetBrains says that it 
pulled out all the stops, rebuilding the compiler 
from the ground up and moving it to a separate process. The result is that projects can 

be compiled automatically in the background on every change a developer makes, so that 
it can run it almost instantly any time. IntelliJ IDEA also embraces the latest versions of 
Java, including its previews and new features that every developer is eager to try. With 
v.12, JDK 8 already can be enjoyed with code assistance for new syntax, such as lambda 
expressions, method references and default methods. Also in the new edition, support for 
Android is expanded with a new Android UI designer. Aesthetically speaking, IntelliJ IDEA 
12 explores the “darker side” of productive coding by unveiling a stylish new dark look and 
feel, dubbed “Darcula”. JetBrains says that Darcula makes the interface clearer and more 
functional, minimizing distraction so users can focus more on the code and less on the IDE. 
http://www.jetbrains.com/idea 


Please send information about releases of Linux-related products to newproducts@linuxjournal.com or 


New Products c/o Linux Journal, PO Box 980985, Houston, TX 77098. Submissions are edited for length and content. 
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Front-End 


Developer 
Should Know about 


Experienced with front-end 
development but new to 
Drupal? Good with Drupal 
configuration but shy 
about theming? Read 
on to find out how to 
get started with custom 
themes, working with 

templates, JavaScript 

and CSS, considerations 
for mobile and tips for 
performance. Plus, learn 

what’s next in Drupal 8. 

ALEXANDER CASTILLO 
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rupal allows you to set up 
a site in no time and lets 
you focus on what really 


matters—content. With Drupal, 
front-end developers can eliminate 
most of the boilerplate code and 
jump right in to styling (otherwise 
known as “theming”) fairly quickly. 
Drupal provides you with the base, 
markup and most of the needed 
functionality right out of the box. 
This article, written by a seasoned 
Drupal front-end developer, intends 
to walk you through the must-know 
Drupal front-end development 
techniques and best practices, 
specifically as related to the latest 
stable version, Drupal 7. This article 
covers the basics on how to create 
a custom theme from scratch and 
how to enhance it in different 
ways with JavaScript and Media 
Queries for mobile support. It 
explains the administration interface 
and shows ways to design pages 
using modules—for example, 
Panels. Finally, it covers 
performance issues and offers 
pointers on what to expect in 
the next version, Drupal 8. 


The “Theme” Layer 

There are two ways of theming a 
Drupal site. One, use a base theme 
that provides most layout and 


functionality needs in combination 
with a sub-theme for custom 
styles. The main benefits of using 
a contributed base theme include 
1) theme settings in the administrator 
interface, 2) multi-column layout 
options, 3) predefined print style 
sheets, 4) contextual classes added 
to the markup and 5) cross-browser 
compatibility fixes. 

Or, option two, you can create 
a theme from scratch and control, 
in detail, everything in the theme 
layer and add features as needed. | 
personally like having total control 
of what is in my theme; therefore, | 
always create a theme from scratch. 
| think every project deserves this 
treatment, so, keeping that in mind, 
let’s explore a creating custom 
theme and compare it with a base 
theme, so you can decide your 
preferred approach. 


Creating a Custom Theme 
Creating Drupal themes from 
scratch is easier that it sounds. 
What do you need? A folder and 
an .info file will get you started: 


® Create a folder in /sites/all/themes. 
m Name the folder with the 
name of the theme. The name 


should be all lowercase with 
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Drupal’s API allows for customization of any 
of the page’s areas in a very modular way. 


no spaces—for example, /sites/ 
all/themes/cooltheme. 


mM Create a new document with your 
preferred text editor or IDE. 


m Add some base settings to this 
document (see below). 


® Save the document inside the 
newly created theme folder. 


m Name the document the 
same as the folder plus the 
info extension—for example, 
cooltheme.info. 


Here are the .info base settings: 


name = cooltheme 

description = Description of the theme. 
core = 7.x 

engine = phptemplate 

regions[left] = Left sidebar 
regions[content] = Content 
regions[header] = Header 
regions[footer] = Footer 


Once the theme has these 
components, enable it from the 


administration interface. On the 
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Drupal toolbar, click on appearance 
(URL: /admin/appearance). The new 
custom theme should be listed on 
this page; enable it, and set it as 
the default. 


TIP: Explore other contributed 
themes. See how they do it! 


Working with Templates 
Drupal’s API allows for customization 
of any of the page's areas in a very 
modular way. Customize the page 
wrapper (html.tpl.php), the page 
itself (page.tpl.php), regions within 
the page (region.tpl.php), blocks 
within regions (block.tpl.php), nodes 
(node.tpl.php) and fields within 
nodes (field.tpl.php). Add these files 
to the theme folder if you want to 
override the templates. You can get 
these templates from core modules 
like node, blocks and field. Simply 
copy and paste the templates inside 
the theme's folder, and let the 
overriding begin! 

Note: make sure to clear 
Drupal’s cache when adding a new 
template or editing the .info file. 
On the Drupal toolbar, click on 
Configuration—Performance—Clear 


html.tpl 


page. tpl 1) 


region.tpl 


region.tpl 


Figure 1. Drupal’s File Template System Hierarchy 


and Priority 


all caches (URL: admin/ 
config/development/ 
performance). 

Are you connecting the 
dots already? As shown 
in Figure 1, there are 
many regions defined 
in the .info file. See 
what content displays 
in the regions by going 
to the block page in the 
administration interface. 
On the Drupal toolbar, 
click on Structure—Blocks 
(URL: /admin/structure/blocks). 


Working with CSS 

To get started styling your 
site, create a folder called 
css inside the theme's 
folder. In there, you can 
add all the stylesheets and 
then reference them from 
the .info file by adding the 
following line(s): 


stylesheets[all][] = css/style.css 
stylesheets[all][] = css/layout.css 
stylesheets[all][] = css/etc.css 


TIPS: You can download the Drupal 7 cheat sheet from http://batayneh.me/sites/ 


default/files/Drupal7-theming-cheatsheet_0.pdf. 


Core modules can be found in the /modules/ folder in the root. 
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Do you ever need to do a 
“one-second” CSS change remotely? 
Before when doing that you needed 
to download or pull the code, make 
the edit, and upload or push, right? 
This incredible module called Live CSS 
(http://drupal.org/project/live_css) 
allows you to edit CSS or LESS 
live within the page, and save the 
changes back to the file instantly. 
This tool is very handy when 
developing a Drupal site. There’s no 
need to switch back from the text 
editor or IDE to the browser and 
refresh the page. All of your edits 
happen right away. 


Working with JavaScript 
JavaScript adds behavior and 
interaction to the theme. Popular 
JavaScript libraries like jQuery and 
jQuery UI already come with Drupal. 
To add custom JavaScript code, first 
create a folder called js inside the 
theme’s folder. Then, simply add 
JavaScript files to the js folder and 
reference them in the theme .info 
file by adding the following: 


scripts[] = js/main.js 
Scripts |] = 1s/ete. 71s 


You also can use Drupal’s PHP 
function drupal_add_js in the 


template.php file as follows: 
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<?php 
function example_preprocess html(&$variables) { 
$options = array( 
‘group’ => JS THEME, 
Vv 
drupal_add_js(drupal_get_path('theme', ‘example'). '/script.js', 
$options) ; 
i 


?> 


Once the JavaScript file is 
referenced in the theme, start adding 
behavior and interaction to the 
theme. Best practice is to wrap the 
JavaScript code inside a jQuery closure 
function—something like this: 


(function($) { 
// Javascript code 
pr) KyQuery); 


But wait, doesn’t Drupal’s API 
provide support for JavaScript? Yes, 
it actually provides specific JavaScript 
functions through jQuery. So, you can 
pass information from the PHP code 
and attach Drupal-specific behaviors 
with something like this: 


(function($) { 
Drupal.behaviors.customToggler = { 
attach: function(context, settings) { 
$(".toggler", context) .once('custom-toggler') .click(function() { 


$(this) .next().slideToggle("slow") ; 


= 


eturn false; 


}) .next() .hide(); 
ba 
ie 


}) (jQuery) ; 


If you're using modules that 
include other JavaScript libraries, 
such as MooTools or YUI, use 
jQuery’s $.noConflict() function 
to avoid confusion over the famous 
dollar-sign alias (§$). 

To practice progressive enhancement 
like some of us do, explore Modernizr. 
Adding Modernizr to your project is as 
easy as installing the Modernizr module 
from drupal.org (http://drupal.org/ 
project/modernizr). Modernizr is a 
JavaScript library that detects HTML5 
and CSS3 features in the user's browser. 


Mobile Support 
When optimizing a Drupal Web site 
for mobile, it is possible to create a 
sub-theme with all mobile features. 
This allows control of the experience 
of the mobile site vs. the desktop 
site. Drupal offers a few things to 
make the job easier. For starters, 
modules like ThemeKey will allow a 
switch to the mobile sub-theme when 
the user is accessing the Web site 
from a mobile device. 

Drupal is “technology-agnostic” 
regarding the front-end technology, 
sO making a responsive Web site in 


Drupal is mostly the same as with 
any other platform. Define the media 
queries in the CSS file, or simply add 
them to the .info file as follows: 


stylesheets[all and (max-device-width: 480px)][] = mobile.css 


Other Drupal modules like Adaptive 
Images will play nicely with Drupal’s 
Image Styles module to create, 
cache and deliver device-appropriate 
re-scaled versions of the site’s 
embedded images automatically. 


Working with the Drupal 
Administration Interface 

Now that you have a custom theme 
in place, let’s look at what the Drupal 
administration interface offers to 
front-end developers. Repetitive 
development tasks like applying 
custom filters to images, embedding 
custom fonts and defining custom 
layouts have never been easier than 
they are with Drupal! 


Image Styles 
The image styles module comes with 
Drupal out the box. With this module, 
you can create image instances of the 
original image and apply effects to 
those instances dynamically—no more 
Photoshop batch scripts; let Drupal do 
it for you, on the Web. 

Effects include: 
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EFFECT 


+ Scale 800x600 (upscaling allowed) 


Ba Select anew effect [Ur 
Crop es 
Desaturate L 

| Resize 
u Rotate 
Scale 
Scale and crop | 
oe 


Configuration—Image styles 
(URL: admin/config/media/ 
image-styles). 


@font-the-face Module 
This module provides an 
administrative interface for 
browsing and applying Web 
fonts embedded via CSS 
from a variety of sources 
and font providers, such as 


Figure 2. Drupal Core’s Image Styles Module 
Administration Screen for Presets 


@ Crop 
m@ Desaturate 
m Resize 
™ Rotate 
m Scale 
™ Scale and crop 

Other effects, such as 
watermarking, overlays, brighten/ 
darken, rounded corners and 
many more, are available via the 
ImageCache Actions contributed 
module (http://drupal.org/project/ 
imagecache_actions). 


From the Drupal toolbar, click on 
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the following: 
m Adobe Edge Web Fonts 
@ Font Squirrel 
m Fontdeck 
® Fonts.com 
™ Google Fonts 
™ Typekit.com 

Here's the link: http://drupal.org/ 
project/fonttheface. It also is possible 
to import local fonts in all Web 
formats: EOT, TTF, WOFF and SVG. 
Icon Fonts Module 
What about the idea of managing 
the icon fonts from the administration 


interface? With this module 
(http://drupal.org/project/iconfonts), 


Alegreya SC Alegreya SC Alegreya SC Alegreya SC Alegreya SC Alegreya SC Alegreya SC 
700 (latin) 700 (latin- 700italic 700italic 900 (latin) 900 (latin- 900italic 
Enable Enable Enable Enable Enable Enable Enable 
Alegreya SC Alex Brush Alex Brush Alfa Slab One Alice regular Alike Angular Alike regular 
900italic 400 (latin) 400 (latin— 400 (latin) (latin) regular (latin) (latin) 
AaGce CthGg  CttGg  BRaGg- AaGg AaGg AaGg 
Enable Enable Enable Enable Enable Enable Enable 
Allan bold Allerta Allerta Stencil Allura 400 Allura 400 Almendra 400 Almendra 
(latin) regular (latin) regular (latin) (latin) (latin-ext) (latin) bold (latin) 
~ 
Aatg AaGg AaGg AaGgy AaGg Hag AaGg 
Enable Enable Enable Enable Enable Enable Enable 
Almendra SC 
400 (latin) 


Enable 


E22 4 3 me 7 i OB 


next> last» 


Figure 3. Google Fonts Drupal Module Administration Screen for Fonts 


you can do so. Install this module 
to get started with the font icon at 
IconMoon (http://icomoon.io) and 
bring it to Drupal in no time. 


Google Fonts Module 

Those familiar with Google Fonts 
know how convenient it is to load the 
custom fonts in a site with a couple 
lines of code. Well, with Drupal, there 
is no need to to use those lines of code 
any more. Install the Google Fonts 
module, enable the desired fonts, 
map it to the CSS selectors from the 
Drupal administration interface and 
enjoy: http://drupal.org/project/ 
google_fonts. 


Panels Module 
If you like the idea of being able to 
create customized page layouts and 
manage the content with a nice-and- 
neat drag-and-drop interface, the 
Panels module is for you. Explore it at 
http://drupal.org/project/panels. 
With Panels, you can do the following: 


™ Create customizable and re-usable 
layout for pages and blocks. 


® Insert the previously created 
content in any region. 


@ Insert and create re-usable custom 
content in any region. 
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Title 
Atechnology company inspired by creative vision. 


Show layout designer Reuse layout 


View: Case Studies: Our Work thumbnails (home page) 2 


» View information 


Insights 


View: Insights: Last entry (home page) a 


» View information 


Update and preview 


Update 


Update and save 


The title of this panel. If left blank, a default title may be used. Set to No Title if you want the title to actually be blank. 


Page title a 


3 Carousel 


View: Homepage: Home image carousel (home page) a 


* View information 


Page Title 
* page title 
: Welcome 
* Homepage 
Our Work About Us 
Services Solutions 


» View information » View information 


; Meet The Team 


View: Profiles: Profile thumbnails (home page) 2 


» View information 


» About Us 


Figure 4. Panels Drupal Module Administration Screen for Content 


™ Customize the node pages and 
node forms at the field level. 


™ Customize the user pages and user 
forms at the field level. 


m Edit and modify the order of 


the panes at all times with drag 
and drop. 
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m Add conditional rules to hide or 
show the panes. 


™ Switch panels based on context— 
for example: user role, is mobile 
and so on. 


m Add CSS classes and IDs. 


™ Cache your panes individually. 


CACHING 


( Cache pages for anonymous users 
© Cache blocks 
Minimum cache lifetime 


<none> + 


Cached pages will not be re-created until at least this much time has elapsed. 


Expiration of cached pages 
1 min . 


The maximum time an external cache can use an old version of a page. 


BANDWIDTH OPTIMIZATION 


External resources can be optimized automatically, which can reduce both the size and number of requests made to your website. 
i Aggregate and compress CSS files. 


4 Aggregate JavaScript files. 
jQuery compression level 
O Production (minified) 


@ Development (uncompressed) 


jQuery and jQuery UI CDN 

O None 

®@ Google 

O Microsoft 

Use jQuery and jQuery U! from a CDN. If the CDN is not available the local version of |Query and jQuery U! will be used. 


Figure 5. Drupal Core’s Performance Administration Screen for Caching, File 
Aggregation, jQuery and jQuery UI CDN 


m Export the configuration as PHP code. the site’s performance. Let’s start with 
Drupal core’s performance settings. 


Performance From the Drupal toolbar, click on 
Now that the theme is ready and Configuration—Performance (URL: 
configurations on the administration admin/contig/development/performance). 
interface are complete, there are a On this screen, activate caching for 
few things you can do to optimize anonymous users and blocks, CSS and 
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JavaScript aggregation and jQuery 
and jQuery UI file compression and 
CDN options. By enabling caching 
and aggregation alone, the site 
performance increases significantly. 
Furthermore, modules like Varnish, 
Boost and Memcache will offer more 
robust caching solutions. And, it 
is recommended to use a CDN in 
combination with the CDN module 
when high-performance is in order. 
Note: for performance testing and 
analysis, visit: http://yslow.org. 


The Future of the Drupal Front End 
How is Drupal moving forward, and 
in what areas will it benefit front-end 
development? There are three major 
initiatives for Drupal 8 that front-end 
developers should be very excited 
about: Design, HTML5 and Mobile. 
The Drupal community has been 
improving the user experience 
constantly, and for version 8, they plan 
to take it another step further. As for 
HTML5 and mobile, the goal is to make 
HTML5 the default output and leverage 
it to make Drupal a fully mobile 
platform. Other unofficial initiatives 
like “Spark” may bring a responsive 
layout builder to Drupal 8. Stay tuned! 
So, what have you learned? 
Although there are many benefits 
to using an out-of-the-box theme, 
it’s worth it to explore creating a 
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custom theme to have control of 
everything in the theme layer and 
features as needed. Take advantage 
of template overrides to craft 
markup, add custom CSS classes, 
and add behavior and interaction to 
pages with JavaScript. For mobile, 
you can create a sub-theme for 
mobile support or just simply use 
CSS3 Media Queries in support of 
responsive design. Before importing 
a third-party library in the theme, 
check whether a Drupal integration 
is available. Most likely a module 
already exists! Leverage design 
modules like image styles, custom 
fonts and panels. For performance, 
be sure to enable caching and 
aggregation, explore contributed 
caching modules and think about 

a CDN. Finally, get excited about 
the future. Drupal 8 will make 
everything even better for cross- 
device optimization. If you have 
any questions, please let me know: 
alex.castillo@door3.com.@ 


Alexander Castillo is the Manager of Front-End Development, a 
solutions architect and an essential part of the Drupal team at 
DOORS. He has been building Web sites and Web applications 
since age 15 and is both a user-experience designer and 
developer. His languages and technologies include: HTML5, CSS3, 
JavaScript, jQuery, AngularJS, EmberJS, Adobe Design Suite, PHP 
and Drupal. He also specializes in cross-browser, cross-platform 
compatibility, responsive design and mobile Web development. 
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FEATURE Using Salt Stack and Vagrant for Drupal Development 


Using 
SALT STACK 
and VAGRANT 
tor DRUPAL 
DEVELOPMENT 


With Vagrant and Salt Stack, you 
can have identical development 
environments and share them 
across local, development, stage 
and production using small text 
configuration files. 


BEN HOSMER 
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hat if, just like Bill Murray 
in Groundhog Day, 
you could wake up to 


a fresh and identical development 
environment completely free of 
yesterday's experiments and mistakes? 
Vagrant lets you do exactly that. 

Or, what if, like Jake Epping in 
Stephen King’s 77/22/63, you could 
make changes and script the past 
without fear, play around with some 
new Drupal modules, and quickly reset 
everything just by leaving and then 
walking back down the stairs of the 
pantry again? 

Would you like to automate the 
creation and installation of a pristine 
Drupal environment, instead of 
manually installing Apache, PHP and 
all of the needed libraries? 

Recently, | read a post from Treehouse 
Agency titled, “End ‘Works on My 
Machine’ Surprises with Vagrant” 
(http://treehouseagency.com/blog/ 
steven-merrill/2011/11/02/end- 
works-my-machine-surprises-vagrant), 
which is about using Vagrant and 
Puppet for Drupal development. You 
may have seen it as well on Drupal 
Planet (http://planet.drupal.org) 
and wondered about the benefits 
described there. 

This is a great technique that 
outlines using various tools to build a 
consistent development environment 


quickly that is repeatable and shareable 
with everyone on your team. 

Linux Journal also recently featured 
an article that introduced Vagrant 
(http://www.linuxjournal.com/ 
content/introducing-vagrant). 

After you're finished here, | urge you 
to go read that one too, because it 
offers more general information about 
Vagrant itself. 

Salt Stack (http://saltstack.org) 
is a tool similar to Puppet, but it’s 
powered by Python, instead of Ruby. 

If you are interested in standalone 

Salt installation and use, check out 

my November 2012 LJ article on that 
topic (http://www.linuxjournal.com/ 
content/getting-started-salt-stack- 
other-configuration-management- 
system-built-python), where | 
introduce Salt Stack and show how to 
install it to control one or thousands 
of other machines. | won't go in 
depth into Salt’s installation here, 
because Salty-Vagrant takes care of 
all of that for you. 

I've been using Salt, Vagrant 
(http://vagrantup.com) and a 
Vagrant gem called Salty-Vagrant 
(https://github.com/saltstack/ 
salty-vagrant) for Drupal 
development quite a bit lately. 

I've found that having a standard 
configuration that mirrors my 
development, testing, staging and 


WWW.LINUXJOURNAL.COM / MARCH 2013 / 85 


FEATURE Using Salt Stack and Vagrant for Drupal Development 


production environments has streamlined 
my workflow and helped prevent a lot of 
the unknowns between different stages 
of the Drupal development life cycle. 
I've been able to minimize a lot of the 
errors and headaches that come with 
integrating multiple software stacks 
and speed up my workflow too. 

| also enjoy being able to try new 
things quickly and easily without 
spending hours reconfiguring and 
re-installing the entire software stack 
needed for Drupal development. 

| then can share these 
configurations quickly and easily 
with other developers on my team 
and know that we are each working 
with the same versions of software 
already installed. In this article, | 
describe how you can do that too. 

With Salt Stack, you store your 
configurations in text files, similar 
to Puppet. The difference is that 
the configurations rely on YAML 
(http://yaml.org) templates instead 
of Ruby code. This makes them easy 
for humans to read, even if they 
aren’t Python coders. 

These configurations can be used 
to launch development machines, 
as well as testing, staging and 
production machines—and all 
of them are identical and easily 
transportable. | use Git to manage 
my team’s configurations. This 
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brings all of the great aspects 
of version control to my server 
configurations too. 

With Vagrant, you easily can 
delete a development environment 
and then re-create it in about as 
much time as it takes for the server 
to boot and install the packages and 
software you specify—kind of like a 
quick undo or reset button. 


Getting Started 

You need to download and install a 
few things to get all of these pieces 
to work together. Each component 
has very good documentation on-line 
for installation and use, so | won't 
duplicate the instructions here. 
Instead, follow the links for each 
piece of software and check the 
projects’ sites for the most up-to- 
date information. 

This article focuses on using an 
Ubuntu-based 64-bit VirtualBox, 
but Salty-Vagrant with Salt Stack 
supports BSD, Red Hat, OS X and 
Windows virtual machines as well. 

The beauty of using Vagrant with 
Salty-Vagrant Is that the configuration 
files, also known as SLS files, transfer 
exactly to production servers, as well 
as anywhere else you might use them. 
The benefit of creating one template 
and then re-using it again and again 
can help you direct more energy 


toward Drupal development, instead 
of troubleshooting server-software 
installation too. 

You need to do this installation and 
configuration only once. After that, it 
is as Simple as vagrant up! 


Install VirtualBox 

VirtualBox allows you to run a guest 
operating system inside your own 
host operating system. It is similar to 
dual-booting, except you still can use 
your main operating system and have 
access to additional running operating 
systems as well. For example, you 

can run an Ubuntu server inside your 
Windows, Mac or Linux host operating 
system and still access your e-mail 
and text editor from your native host 
operating system. 

Follow the instructions at 
http://virtualbox.org to install 
VirtualBox. Versions are available 
for most major operating systems. 
A major portion of popular Linux 
distributions also have packages 
available through their respective 
package managers. This is the most 
effective and efficient way to install 
the majority of tools | describe here. 


Install Vagrant 

Vagrant is a Ruby gem that allows 
you to create and destroy virtual 
machines quickly, and then reset 


them almost instantly. 

Gems are like Drupal modules, but 
for the Ruby programming language. 
These plugins let you extend Ruby’s 
functionality and add additional 
capabilities. You should ensure that 
you have a recent version of Ruby 
installed. | recommend using your 
package manager if you don’t have 
it, but a lot of distributions include 
Ruby already, so you likely won't 
need to install it. You easily can find 
out whether you have Ruby installed 
by typing ruby --version from 
your terminal. 


Install Salty-Vagrant 
Salty-Vagrant is another Ruby gem 
that brings the functionality and 
automated configuration tools of Salt 
Stack to Vagrant. Salty-Vagrant acts 
as a bridge between the Salt Stack 
configuration management system 
and Vagrant/VirtualBox. The project's 
page has installation instructions 
that are updated on a regular basis. 
It would be a good idea to refer 

to these instructions just in case 
something has changed after this 
article is published. If you know a 
little Ruby or shell scripting, feel free 
to join in this open-source project 
and help contribute back. You'll find 
the development community to be 
friendly and welcoming. 
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Get the Base Box 
Get the base box from 
http://vagrantbox.es. You'll need a 
box in which to create and run your 
environment. Boxes are what Vagrant 
uses for your virtual machines, and 
they are simply virtual machines that 
have been configured specifically 
with Vagrant support. You can create 
your own from scratch, but it is much 
quicker and easier just to download 
a pre-made one that has all of the 
necessary Vagrant configurations 
already set up. | used the precise64 
box. This is an Ubuntu 12.04 64-bit 
Linux server with the necessary Vagrant 
tools and guest additions already in 
place. You're downloading an entire 
operating system, so be patient, 
because it might take a little while. It 
usually is best to download your box 
and then save it locally somewhere so 
you Can use it again. 

If you're using a 32-bit host 
operating system, you should instead 
download the 32-bit version. 


Import the Box 
Hopefully by this point, you have 
everything you need. Let’s use the 
command line to interact with Vagrant 
and import your new box. 

To import your box, use: 


vagrant box add precise64 /path/to/your/downloaded/box 
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Relative path names don’t work well 
here, so if you downloaded your box 
to your home directory, you need to 
type out the full path—for example, 
~/bhosmer/precise64.box should 
be typed out like this: /home/ 
bhosmer/precise64. box. 

The vagrant box add command 
creates a copy of your box and 
stores it so that Vagrant can access 
it. This box is used to create the 
instantiation of the particular box 
you happen to be using. The nice 
part is that you always can press the 
“reset” button by using vagrant 
destroy to delete anything you've 
done and set it back to Its original 
state before you started. | go into 
that in more detail a little later. 


Launch the Box and Verify the 
Guest Additions 
Now, launch your box just to make 
sure that the VirtualBox guest 
additions are up to date. If they aren't, 
Salty-Vagrant may not work properly. 
Where you store your Vagrant 
configurations is entirely up to you. 
| like to keep mine in my home 
directory under a vagrant folder, just 
so | can keep track of the different 
environments | have. 
Create a new directory called salty- 
vagrant and cd into that directory. 
Now use vagrant init to initialize 


a Vagrantfile there. This is a standard 
Vagrant configuration file that is 
well documented. You won’t need 

to change any of the configurations, 
except for line 10: 


config.vm.box = "base" 


If you followed the quick- 
start instructions from 
http://www.vagrantup.com, 
Vagrant specifies your default boxes 
as base. Change this to the name of 
the box you imported, precise64: 


config.vm.box = "precise64" 


Notice that | left the .box 
extension off. 

Now that your Vagrantfile points to 
the correct box, use vagrant up to 
start your virtual machine. 

If you get a warning that your 
guest additions are out of date, 
and it is quite likely that you will 
if you just installed VirtualBox, 
proceed to the next step. If you 
didn’t, you can skip to the section 
“Add a Salty-Vagrant Vagrantfile”. 


Updating the Guest Additions 

If you do need to update the guest 
additions installed on your box, in 
this section, | explain how to do it 
on the Ubuntu-based box you just 


downloaded. This will be one 

of the few times that your changes 
will be permanent within a box. 
And this time, you want them to 
be. The guest additions allow your 
host operating system to interact 
with the guest operating system. 
This includes sharing files and 
network resources between them. 
You store your configuration files 
on your host operating system, 
and then they are copied to the 
guest operating system when 

you start it up each time, so it is 
important that the shared-folder 
functionality works. 

After Vagrant finishes booting the 
guest operating system, you'll need 
to connect to this box using SSH. 
Use the Vagrant-supplied command: 
vagrant ssh. 

You then will be logged in to 
the guest operating system as the 
“vagrant” user. 

Updating the guest additions 
probably is the trickiest and most- 
involved task. If you run into any 
problems, lots of resources are available 
on-line. Most distributions mention a 
number of solutions in their respective 
forums, and the VirtualBox Manual 
(https://www.virtualbox.org/ 
manual/ch04.html) also has some 
detailed information. I’m going to 
show you how to download and 
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update the virtualbox guest additions 
from virtualbox.org, and update them 
on the precise64 box that you just 
downloaded as an example. 

First, you need some packages to 
build and compile the Linux kernel. 
Use Ubuntu’s package manager, apt, 
to install them: 


sudo apt-get install dkms linux-headers-3.2.0-23-generic 


If you're using a non-Debian 
machine, search for the proper 
names in your particular package 
manager and use the header version 
that matches your particular kernel. 

With kernel support now 
in place, browse to 
http://download.virtualbox.org/ 
virtualbox and locate the folder 
that matches the version of VirtualBox 
that you have installed. Generally, 
you can find what version you 
have through the VirtualBox 
graphical interface or the 
warning message generated by 
Vagrant when you booted your 
virtual machine. Now, locate the 
VBoxGuestAdditions_x.x.x.iso that also 
matches the version of Virtualbox that 
you have installed. Replace the x in 
the filename with the specific version 
for your version of VirtualBox. 

You can copy the URL to this file 
and paste it into the terminal window 
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of your guest Ubuntu system that you 
had previously SSH’d into. Use wget 
to download the ISO: 


wget http://download.virtualbox.org/virtualbox/x.x.x/ 


=»VBoxGuestAdditions_x.x.x.iso 


Again, replace the x in the filename 
with the version that matches your 
VirtualBox installation. 

You now should have an .iso file in 
your home directory. 

After it’s downloaded, mount it so 
that you can access the files contained 
in it by first creating a temporary 
directory in which to mount this 
iso. Within your home directory, 
use this command: 


mkdir -p tmp/vbox 
Now, mount the .iso into that folder: 


sudo mount -o loop VBoxGuestAdditions_x.x.x.iso tmp/vbox/ 


Within the .iso, you'll find a 
VBoxLinuxAdditions.run script. Copy 
this to your home directory: 


cp VBoxLinuxAdditions.run ~ 


Now, make it executable by adding 
the execute flag: 


chmod +x VBoxLinuxAdditions.run 


And finally, run the installation script: 


sudo ./VBoxLinuxAdditions.run 
If you get a warning about 
failing to install the window system 
drivers, you safely can ignore it. The 
box you are installing these guest 
additions on is a server and doesn’t 
have a window manager installed. 
You also can use the --nox11 
flag like this when you install the 
updated guest additions: 
sudo ./VBoxLinuxAdditions.run --nox11 
The last step in this installation 
is to reboot the machine to ensure 
that the guest additions are up to 
date with: 


sudo reboot 


After the box restarts, log back 
in with: 


vagrant ssh 


Now you can unmount the 
SO USING sudo umount /mnt, 
if it isn’t already, and delete the 
VBoxLinuxAdditions.run and the .iso 
file to save space. You also optionally 
can do some housekeeping by 
deleting the .bash_history file. 


Package the Updated Box 
Now that the guest additions are 
updated, you should repackage a new 
box so the guest additions match each 
time you turn on this Vagrant box. 
This keeps you from needing to update 
them every time. As long as you don’t 
change the version of VirtualBox that 
you are using, you won’t need to 
update your guest additions. If you do, 
you'll need to update them again. 
Using the vagrant package 
command, export your new box. This will 
shut down your machine and export a 
package.box file in the current directory. 
Rename this to something more 
descriptive, like myprecise64.box. 
Now add this new box to Vagrant: 


vagrant box add myprecise64 myprecise64.box 


The names you give your boxes are 
arbitrary and entirely up to you. Feel 
free to use a description that makes 
sense to you. 

You're done with the original box 
now, so destroy it with vagrant 
destroy. You can remove it entirely 
from Vagrant with vagrant box 
remove precise64. 

Add your new myprecise64 box, or 
whatever you happened to call it, like 
you did previously with the base box you 
downloaded with vagrant box add. 

Make sure to change the name in 
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your Vagrantfile to match the new 

box, and test it with vagrant up. 
Congratulations! You now 

have an updated Ubuntu 12.04 

VirtualBox suitable for Salt Stack 

and Salty-Vagrant automation. 


Install the Vagrant Salty-Vagrant 
Gem 
The final piece of software you'll need to 
install is the Salty-Vagrant gem. It allows 
the automatic installation of Salt Stack on 
your guest virtual machines and uses Salt 
Stack to initiate your configuration and 
installation of software on them as well. 
Back on your host machine (that is, 
the one that is not your virtual machine), 
use the Vagrant gem command vagrant 
gem install vagrant-salt to 
install the Salty-Vagrant gem. 


Add a Salty-Vagrant-Specific 
Vagrantfile 

Now you need to add a slightly 
customized Vagrantfile to configure 
some Salt-specific parameters, share 
your state tree and automatically install 
Drupal and the Linux Apache MySQL 
Stack after your VirtualBox is started. If 
you added a Vagrantfile previously, you 
can delete it now. You’re going to add 
a smaller, more customized one that Is 
available from the Salty-Vagrant project 
page at https://github.com/saltstack/ 
salty-vagrant. You simply can copy 
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and paste this into a new Vagranttile. 

Locate the line config.vm.box = 
"precise64" and change "precise64" 
to the name of your box. If you've been 
following along, earlier in the tutorial, | 
exported my box as "myprecise64". 

| also like to add guest/host network 
communications with this line to my 
Vagrantfile too: 


config.vm.network :hostonly, "192.168.33.19" 


You can use any available IP 
address for your internal VirtualBox 
network. Keep in mind, this is 
separate from your own network and 
is created by VirtualBox. This allows 
you to access your Drupal site from 
your own operating system's Web 
browser using the IP address that you 
specified. You can choose another IP 
address if you want. 


Create Your Minion Configuration 
File 

In the same directory as your Vagrantfile, 
create a salt directory, srv directory and 
lamp-drupal directory like this: 


mkdir -p salt/srv/lLamp-drupal 


Create a new minion.conf file in your 
salt directory that contains this line: 


file_client: local 


This file instructs Salt that when the machine is 
booted to install all of the needed packages for 
running a Drupal Web server. I’ve even included a 
Drush installation. 


This instructs Salt not to search 
for a master server, but instead to 
use the local file structure for its 
configuration information. This 
is what you'll see referred to in 
the Salt Stack documentation as a 
masterless configuration. 

Locate the line in your Vagrantfile 
labeled salt.minion_conf and 
uncomment it by removing the #. 
This tells Vagrant where your minion 
configuration is located. The path is 
relative to your Vagrantfile. 


Get the State Files 

My Drupal SLS file is available in the salt- 
states repository (https://github.com/ 
saltstack/salt-states/tree/master/ 
small/lamp-drupal). You can use this 
to install all of the needed packages 

in Ubuntu for Drupal development. 
Download that file now and place it 

in your /srv/salt/lamp-drupal directory. 
You'll also need to create a /srv/salt/top. 
Sls file that looks like this: 


base: 


Voae hs 


- Lamp-drupal 


For more details about the top file, 
see my previously mentioned article 
about installing Salt Stack. 


Add the State File to Your Shared 
Folder 
Within the salt/srv/lamp-drupal 
directory, copy the init.sls file from 
the salt-states repository. 

This file instructs Salt that when 
the machine is booted to install all of 
the needed packages for running a 
Drupal Web server. I’ve even included 
a Drush installation. If you examine 
the file, you'll find that it is fairly 
understandable already. | won’t go 
into much more detail here, but feel 
free to take a look at the init.sls file to 
see what actually is being installed. 

Now edit your Vagrantfile to reflect 
the location of this state file by 
editing the line: 


config.vm.share_ folder "salt_file root", "/srv", 


="/path/to/salt_file_root" 
to; 


config.vm.share_folder "salt_file_root", "/srv/salt", "salt/srv" 
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Vagrant Up! 

With your minion configuration file in 
place and your state file ready to go, 
this last step will start your machine 
automatically, install Salt and install all 
of the necessary components needed 
for a Drupal development server. 

Ready? Use vagrant up to start 
your machine. 

You can watch the output and 
responses as Vagrant, Salty-Vagrant 
and Salt Stack automatically download 
all of the software you specified and 
configure it for you. 

After the server is started, simply 
SSH to your virtual server using 
vagrant ssh again. You'll find a 
/var/www folder ready to receive your 
Drupal installation. 

This SLS file is reusable across 
multiple environments, and | 
haven't configured it to change the 
permissions of the /var/www folder. 
You'll notice this folder is owned by 
root, just like the Ubuntu package 
manager sets it. 

The Drush directory located within 
the vagrant user’s home directory is 
also owned by root. You'll get an error 
with Drush if you don’t change the 
ownership of that folder to the user 
that is initiating the Drush command. 


Conclusion 
By combining four very powerful tools: 
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Vagrant, Salty-Vagrant, VirtualBox and 
Salt, you can speed up your Drupal 
development considerably. Maintaining 
small development environments and 
then scripting their configuration helps 
create a win for everyone involved 

by adding consistency across all your 
developers’ platforms. Maintaining 
small, text-based configuration files 
can save money on storage costs too. 
You and your developers also can save 
time transferring new configurations 
to remote servers. 

You can try out new configurations 
and install additional libraries 
if you need or want to, without 
fear of breaking your local or 
production systems. 

Each of the tools | have outlined 
are available for no cost and are 
open-source projects. You'll find 
the Salt development community to 
be one of the friendliest and most 
welcoming communities around the 
Open Source world. 

Each project is updated on a regular 
basis, and it would be a good idea 
to review the documentation, as well 
as changes to the software, if you 
encounter any problems.m™ 


Ben Hosmer is a Drupal Developer and DevOp with 
RadiantBlue Technologies. He has made contributions to 
Drupal as well as Salt Stack. He is the maintainer of the 
Classified Drupal module, and he enjoys teaching Drupal. 
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avaScript has had a long- 

standing monopoly on client- 

side Web programming. It has a 
tremendously large user base, and 
countless libraries have been written 
in it. Surely it is the perfect language 
with no flaws at all! Unfortunately, 
that is simply not the case. JavaScript 
is not without its problems, and there 
exists a large number of libraries 
and “trans-pilers” that attempt to 
work around JavaScript’s more quirky 
behaviors. JQuery, Coffeescript, 
Objective-J and RubyJS are examples 
of how people have been trying to 
make JavaScript better. However, a 
new contender is throwing its hat into 
the ring in a big way. 

In comes Google Dart, a new 
JavaScript replacement language. 
Dart is a ground-up re-imagining of 
what JavaScript should be. It requires 
its own runtime environment (which 
Google makes available for free under 
the three-clause BSD license) and has 
its own syntax and libraries. Dart is 
an object-orientated language with a 
heavy Java-like feel, but it maintains 
many of the loved JavaScript 
paradigms like first-class functions. 

So, why have | chosen to use Dart? 
Good question! | have chosen Dart 
because it is a clean break from 
JavaScript, and it has the object- 
orientated programming style | have 


come to know and love. Because | 
have a Java background, the learning 
curve didn’t seem as steep with 

Dart as it does with JavaScript. Also, 
because Dart Is so new, it gives me a 
chance to become an early adopter of 
the language and watch its evolution. 


Installing Dart 

Before you can program with Dart, 
you need to grab a copy from 
http://dartlang.org. | chose to install 
only the SDK; however, there is an 
option to grab the full Dart integrated 
development environment with the 
SDK included. Eclipse users will feel 
right at home with the IDE, because it 
is based on Eclipse components. 

To install the SDK, | just unzipped 
the files and copied the whole 
directory to $HOME/bin. Then | 
modified my path variable to look 
in the folder | created: 


PATH=$PATH: $HOME/bin/dart-sdk/bin 


Now | can run dart, dart2js and 
dartdoc from anywhere. 


Language Features 

The core Dart language is pretty 
straightforward. The basic data 
types available are var (stores any 
object), num (stores any number 
type), int, double, String, bool, List 
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(arrays) and Map (associative array). 
All of these data types are declared 
in the dart:core library. dart:core is 
always available and does not need 
to be imported. Functions also can 
be considered a data type, because 
Dart treats them as first-class objects. 
You can assign functions to variables 
and pass them as parameters to 
other functions or write anonymous 
functions in-line. 

For flow control, you have the 
“usual” if, else if, else, for, while 
and do-while loops, break, continue, 
switch and assert. Exceptions are 
handled through try-catch blocks. 

Dart has a lot of support for 
object-oriented programming. 
Classes are defined with the class 
keyword. Every object is an instance 
of some class, and all classes 
descend from the Object type. Dart 
allows only for single inheritance. 
The extends keyword Is used to 
inherit from a parent class other 
than Object. Abstract classes can 
be used to define an interface 
with some default implementation. 
They cannot be instantiated 
directly, but can make use of 
factory constructors to create the 
appearance of direct instantiation. 
Abstract classes are defined with 
the abstract modifier in front of the 
class declaration. 
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Standard Library 

Dart ships with an impressive standard 
library. | use a few of the libraries and 
classes in my examples here, so it will 
be helpful to have an idea of what 
they can do ahead of time. | can’t 
cover all of them, but | cover the ones 
| personally find most useful. 

As | said earlier, dart:core defines 
all of the core data types that are 
available. That’s not all though! 
dart:core also contains the regular 
expression classes that are an 
invaluable addition to any standard 
library. Dart uses the same regular 
expression syntax as JavaScript. 

dart:io provides classes that let your 
program reach out to the world. The 
File and Directory classes can be used 
to interact with the local filesystem. 
The File class also will allow you to 
open input and output streams on 
the specific file. If you want to write 
cross-platform code and allow users 
to specify the path of a file native 
to their operating system, the Path 
class provides a really nice 
Path.fromNative(String path) 
constructor that will convert Windows 
and UNIX paths to their Dart 
counterparts. Also included in this 
library are the Socket and ServerSocket 
classes that can be used for traditional 
network communications. The 
HttpServer class allows you to program 


a Web server quickly. This is great if 
you want to add a custom rest API 
to your application. dart:io can be 
imported and used only on server-side 
applications, so don’t try to use it in 
your browser apps! 

dart:html contains all of the 
classes necessary to interact with 
a client browser’s document object 
model. This library is required to 
write any client-side code that runs 
in a browser. The library defines 
two static methods: Element 
query(String selector) and List<Element> 
queryAll(String selector). These 
methods allow you to grab HTML5 
elements from the browser's DOM 
using cascading-stylesheet selectors. 
(| show an example of this later.) 

dart:math, dart:json and dart:crypto 
provide helpers that are hard to 
live without. dart:math provides 
all of the static math methods that 
programmers have come to expect. 
dart:json provides the JSON helper 
class. It has only three static methods: 
parse(String json), which returns a 
Map containing the parsed document; 
String stringify(Object object) and void 
printOn(Object object, StringBuffer 
output) can be used to serialize an 
object into JSON. Any object can be 
made serializable by implementing 
a toJson() method. dart:crypto has 
helpers for performing md5, shat 


and sha256 hashing. There also is a 
CryptoUtils class with methods for 
converting bytes to hex and bytes 
to base64. 


Server-Side Programming 
Let’s jump into Dart by looking at 
some server-side programming: 


import ‘dart:io'; 


void main() { 
String fileName = './test.txt'; 
File file = new File(fileName) ; 


var out = file.openOutputStream() ; 
Out. writeString("Hello, Dart! \n"); 
Out.close(); 


Does it look pretty familiar? It’s not 
much different from a Java program at 
this point. You start by importing the 
dart:io library. This gives you access 
to the File and OutputStream classes. 
Next, you declare a main method. 

Just like Java and C, main acts as the 
entry point for all programs. The File 
object is used to hold a reference to a 
single file on the filesystem. In order 
to write to this file, you open the file's 
output stream. This method will create 
the file if it does not exist or clears 
the contents of the file if it does. It 
returns an OutputStream object that 
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then can be used to send data to the 


file. You write a single string and close 


the OutputStream. 


Listing 1. wunder.dart 


import ‘dart:io' 
import ‘dart:uri'; 


import 'dart:json'; 


void main() { 
List jsonData = []; 
String apikey = ""; 
String zipcode = ""; 
//Read the user supplied data form the options 
//object 
try { 
apiKey = new Options().arguments [0] ; 


Zipcode = new Options().arguments[1]; 


oa 


on RangeError { 
print("Please supply an API key and zipcode!"); 
print("dart wunder.dart <apiKey> <zipCode>") ; 


exit(1); 


//Build the URI we are going to request data from 
Uri uri = new Uri("http://api.wunderground.com/" 


"api/${apikey}/conditions/q/${zipcode}. json") ; 


HttpClient client = new HttpClient(); 
HttpClientConnection connection = 

client. getUrl (uri) ; 
connection.onResponse = 


(HttpClientResponse response) { 


//Qur client has a response, open an input 
//stream to read it 
InputStream stream = response. inputStream; 


stream.onData = () { 
//The input stream has data to read, 


//read it and add it to our list 


jsonData.addA1L1(stream. read()); 
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To run this program, save it to a 
file called first.dart. Then use the 
Dart runtime environment provided 


stream.onClosed = () { 


//response and print the location and temp. 
try { 
Map jsonDocument = 
JSON. parse(new String. fromCharCodes(jsonData) ) ; 
if (jsonDocument ["response"] .containsKey("error")) { 
throw jsonDocument["response"] ["error"] ["description"] ; 
} 
String temp = 
=» jsonDocument ["current_observation"] ["temperature_string"] ; 
String location = jsonDocument["current_observation"] 


["display_location"] ["full"]; 


print('The temperature for $location is $temp'); 


oa 


catch(e) { 
print("Error: $e"); 


exit(2); 


//Register the error handler for the InputStream 
stream.onError = () { 
print("Stream Failure!"); 


exit(3); 


//Register the error handler for the HttpClientConnection 
connection.onError = (e){ 

print("Http error (check api key)"); 

print("$e"); 

exit(4); 


with the SDK: 


* dart rst. dart 


When your program Is done, you 
should see a file called test.txt in the 
same directory. Open it, and you will 
see your text. 

What's interesting about Dart is that 
all I/O is event-based. This is much in 
the same style as Node.js. Every time 
you call a method that performs I/O, 
it is added to the event queue and the 
method returns immediately. Almost 
every single I/O method takes in a 
callback function or returns a Future 
object. The reason for this design choice 
is for scalability. Because Dart runs your 
code in a single thread, non-blocking 
asynchronous I/O calls are the only 
way to allow the software to scale to 
hundreds or even thousands of users. 

In Listing 1, you can see this 
evented I/O style put to work with 
the HttpClientConnection 
object returned by the 
HttpClient.geturl (uri url) 
method. This object is working 
in the background waiting 
for a response from the HTTP 
server. In order to know 
when the response has been 
received, you must register an 
onResponse(HttpClientResponse 
response) callback method. | 


created an anonymous method to 
handle this. Also notice that toward 
the bottom of the program, | register 
an onError() callback as well. 
Don’t worry; all of the callbacks 
are registered before the HTTP 
transaction begins. 

Once the onResponse() callback 
is executed, | pull the InputStream 
object out of the response to begin 
reading data. | register the onData(), 
onClosed() and onError() callbacks 
to handle the different states the 
InputStream can be in. onData() 
simply reads bytes off the stream and 
appends them to the jsonData list 
object. onData() is guaranteed to 
be called as long as there is data to 
read. Once the stream has hit “end 
of file”, onClosed() is executed. At 
this point, | know all of the data from 
the HttpRequest has been transferred 
and read, so | can use the JSON 
helper class to parse the response 
into a Map object and print the 
final result to the user. This is where 
the program actually exits from if 
everything was successful. If there 
was an error in the InputStream, then 
the onError() callback would have 
been called, and the program would 
have exited from there. 

To run this program, call it with 
the Dart runtime environment. You 
will need to register for an API 
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key from Weather Underground 
(http://www.wunderground.com/ 
weather/api). Don’t worry; it’s 
completely free. Once you have your 
API key, you can check the current 
temperature for any US zip code: 


$ dart wunder.dart ec7....93b 10001 


The temperature for New York, NY is 57.2 F (14.0 C) 


Client-Side Dart 
Now that you've seen what Dart can do 
on the server side, let’s take a look at 
what it really was designed for, the client 
side. Dart excels at programming large- 
scale browser applications. Unfortunately, 
Space constraints prevent me from 
showing a truly large application. Instead, 
| cover a not-so-large but very cool 
application using the HTML5 Canvas 
object. Let's use Dart for finger painting. 

In our simple finger-painting 
application, there will be buttons for 
each color that is available to users, 
as well as buttons to increment and 
decrement the thickness of their 
strokes. What good is painting a 
masterpiece if you can’t save it and 
Share it with the world? So, let’s make 
a save button that will convert the 
canvas to a PNG image. 

First, let’s take a look at the 
markup for this project. In Listing 3, 
you can see there is an HTML5 Web 
page that contains a canvas element 
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called draw-surface. This is where 

the work of art will be made. Below the 
canvas are the control buttons that allow 
users to select colors and stroke width, 
and the save button. The last part of the 
document Is the most interesting part. 
There are two script elements. The first 
is a script tag with the type attribute 
set to “application/dart”. This script 
type is currently recognized only by a 
fork of Chromium called Dartium 
(http://www.dartlang.org/dartium). 
The second is a JavaScript bootstrap 
file that is required to start the Dart 
VM in Dartium. It also has a special 
second function that | talk about later. 

Now let's take a look at the 
application itself. At the top of Listing 
2, | start the program by importing 
dart:html. All client-side applications 
must import this library to have access 
to the DOM. Next, | create a class called 
DrawSurface that will act as a container 
class for the canvas object. The 
constructor takes in a CanvasElement 
and grabs its 2-D-rendering context. 

It also registers all of the callbacks to 
handle mouse movements on the draw 
surface. When the user presses down 
on the mouse, somewhere on the draw 
surface canvas | begin a draw path. 

As the user moves the mouse 
around with the button pressed down, 
| add line segments to the drawing. 
When the user releases the mouse or 


Listing 2. fingerpaint.dart 


library fingerpaint; 


import ‘dart:htm1'; 


class DrawSurface { 
String _color = "black"; 
int _lineThickness = 1; 
CanvasELlement _canvas; 
bool _drawing = false; 


var _context; 


DrawSurface(CanvasElement canvas) { 
_cCanvas = canvas; 


_context = _canvas.context2d; 


_canvas.on.mouseDown.add((Event e) => _onMouseDown(e) ) ; 
_canvas.on.mouseUp.add((Event e) => _onMouseUp(e)) ; 
_canvas.on.mouseMove.add((Event e) => _onMouseMove(e)) ; 


_canvas.on.mouseOut.add((Event e) => _onMouseUp(e)); 


set LineThickness(int LineThickness) { 
_lineThickness = lLineThickness; 


_context.LineWidth = _lineThickness; 


set color(String color) { 
_color = color; 
_context.fillStyle = color; 


_context.strokeStyle = color; 


int get LineThickness => _lLineThickness; 


int get color => color; 


void incrementLineThickness(int amount) { 
_lineThickness += amount; 


_context.LineWidth = _lLineThickness; 


String getPNGImageUr1l() { 


return _canvas.toDataUrl('image/png') ; 


_onMouseDown(Event e) { 
_context.beginPath() ; 
_context.moveTo(e.offsetX, e.offsetyY) ; 


_drawing = true; 


_onMouseUp(Event e) { 
_context.closePath() ; 


_drawing = false; 


_onMouseMove(Event e) { 
if (drawing == true) { 


_drawOnCanvas(e.offsetX, e.offsetyY) ; 


_drawOnCanvas(int x, int y){ 
“eontext. LineTo(x, ¥)3 


_context.stroke() ; 


void main() { 


CanvasElement canvas = query ("#draw-surface") ; 


DrawSurface ds = new DrawSurface(canvas) ; 


List<Element> buttons = queryAl11("#colors input") ; 
for (Element e in buttons) { 
e.on.click.add((Event eve) { 
ds.color = e.id; 


Hs 


var sizeDisplay = query("#currentsize") ; 


sizeDisplay.text = ds. lineThickness.toString() ; 


query ("#sizeup").on.click.add((Event e) { 
ds.incrementLineThickness (1); 
sizeDisplay.text = ds.lineThickness. toString () ; 
WS 


query ("#sizedown").on.click.add((Event e) { 
ds.incrementLineThickness(-1); 
sizeDisplay.text = ds.lineThickness. toString (); 
ids 


query ("#save").on.click.add((Event e) { 
String url = ds.getPNGImageUrl() ; 
window.open(url, "save") ; 


HS 
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Listing 3. fingerpaint.html 


<!DOCTYPE html> 
<html> 
<head> 
<title>Finger Paint</title> 
<link rel="stylesheet" href="fingerpaint.css" /> 
</head> 
<body> 
<h1l>Finger Paint</h1> 
<div> 
<canvas id="draw-surface" width="800px" height="600px"> 
</canvas> 
</div> 
<div id="colors"> 
<input id="white" type="button"></input> 
<input id="red" type="button"></input> 
<input id="black" type="button"></input> 
<input id="blue" type="button"></input> 
<input id="green" type="button"></input> 
<input id="purple" type="button"></input> 
<input id="yellow" type="button"></input> 
<input id="orange" type="button"></input> 
<input id="brown" type="button"></input> 
</div> 
<div> 
<input id="sizeup" type="button" value="Increase width"> 
</input> 
<input id="sizedown" type="button" value="Decrease width"> 
</input> 
<span id="currentsize"></span> 
</div> 
<div> 
<input id="save" type="button" value="Save"></input> 
</div> 
<script type="application/dart" src="fingerpaint.dart"> 
</script> 
<script type="application/javascript" 
src="http://dart.googlecode.com/svn/trunk/dart/client/dart.js"> 
</script> 
</body> 
</html> 


104 / MARCH 2013 / WWW.LINUXJOURNAL.COM 


Listing 4. fingerpaint.css 


#draw-surface { 


border-style: solid; 


border-width: 2px; 


} 
#white { 


background-color: 


width: 30px; 
} 
#red { 


background-color: 


width: 30px; 
} 
#black { 


background-color: 


width: 30px; 
} 
#blue { 


background-color: 


width: 30px; 
} 
#green { 


background-color: 


width: 30px; 
} 
#purple { 


background-color: 


width: 30px; 
} 
#yellow { 


background-color: 


width: 30px; 
} 


#orange { 


background-color: 


width: 30px; 
} 
#brown { 


background-color: 


width: 30px; 


white; 


red; 


black; 


blue; 


green; 


purple; 


yellow; 


orange; 


brown; 


moves out of the canvas element, | 
close the drawing path. 

| implemented getters and setters for 
the color and lineThickness attributes. 
In the setter methods, | make sure to 
update the rendering context on any 
change. | also add two methods 
incrementLineThickness(int amount) 
that will allow the user to adjust the 
lineThickness by some amount, instead 
of just setting it, and getPNGImageUr1() 
to expose the canvas element's 
toDataUrl() method. This method 
will allow the save button to function. 

In main, | use the static 
query(String selector) function 
to get the canvas element by its ID. 
The query function takes any CSS 
selector and returns an Element 
object that matches it. If there is more 
than one element that you want to 
access On a page, you can use the 
queryAll(String selector) function 
that will return a List<Element> object. 
| use this function to gather up all of 
the color buttons at once and register 
onClick() events to set the current color 
to its respective ID value. 

Finally, | register the callbacks for 
the size-up and size-down buttons 
that change the thickness of the line 
by 1. | also register the callback for 
the save button to grab the PNG data 
URL from the canvas and open it ina 
new window. The user then can save 
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the image by right-clicking on it and you need to grab a copy of Dartium 


choosing “Save image as”. from http://www.dartlang.org/ 
dartium. To install it, | just unzipped 

Running the Application the file and created a symlink in my 

If you have chosen to download and $HOME/bin directory to the chrome 


install the full Dart editor package, you. program that | extracted: 
already have Dartium installed. If, like 
me, you chose to install only the SDK, $ In -s /path/to/unzipped/folder/chrome dartium 


NOTE: Dartium is an experimental browser, so it should be used only for developing Dart 
applications locally. Don’t use it as your normal browser! There might be security exploits 
or stability issues that have not been discovered yet. 


Finger Paint 


©@ | © file:///home/james/projects/dart/fingerPaint/fingerpaint.html 


Finger Paint 


mm | f| | | lhe 


Increase width | | Decrease width | 7 


Save 


Figure 1. The finger-paint application running on Dartium, a special fork of Chromium. 
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Once It is installed, you can run this =Remember that bootstrap script line in 


application from the command line fingerpain.html? Aside from starting the 
with the command: Dart VM in Dartium, it also will fall back 
to a JavaScript application if Dart is not 
$ dartium fingerpaint.html supported. The JavaScript application 
must have the same name as the Dart 
Running on Other Browsers application with the extension .dart.js. 
It's okay if you don’t have Dartium. The Dart SDK comes with a nifty 


©©@®©® Finger Paint - Mozilla Firefox 


; Finger Paint 


File:///home/james/projects/dart/fingerPaint/Fingerpaint.html v@| we Q fay wv 


Ki 


Finger Paint 


ay 


IE REE 


Increase width Decrease width | 3 


Save 


Figure 2. After being converted to JavaScript, the finger-paint application can now run 
on any browser. Here it is running on Firefox. 
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program called dart2js that will convert 
a Dart browser application into a 
JavaScript application for use in any 
browser. To convert this application, you 
can run dart2js on fingerpaint.dart: 


$ dart2js -ofingerpaint.dart.js fingerpaint.dart 


When this is done, you will see several 
new files, including fingerpaint.dart.js. 

Now the application will work in any 
browser that can handle JavaScript. | 
personally recommend using Dartium 
for application development and 
then converting the application to 
JavaScript for release. 


State of Dart 

| would love to tell you that the 
community has welcomed Dart with 
open arms, but that’s simply not the 
case. The people in charge are afraid 
of Dart becoming the next VBScript 
and hurting the open Web. So far, 


Resources 


Dart Home Page: http://www.dartlang.org 


Microsoft, Mozilla and Apple have 
rejected the idea of embedding a Dart 
runtime into their browsers. As Dart 
matures and gains popularity, | hope 
to see this stance reverse, but for 
now, dart2 js is the only way to get 
Dart projects on-line for all to use. 


Conclusion 

Dart is a fantastic language that 
presents an entirely new approach to 
writing large-scale, client-side, object- 
oriented applications. | have enjoyed 
working with it, and | hope you will 
too. The potential of this language 

is limitless, and | hope to see wide- 
spread adoption of it in the future. m™ 


James Slocum is a software developer for Telvue Corporation, 
where he designs and programs digital media decoders 
and delivery systems. In his spare time, he enjoys learning 
new programming languages and snowboarding. Feel free 
to contact him at jms44@njit.edu, and check out his blog 

at http://www.jamesslocum.com. 


Dart API Reference: http://api.dartlang.org/docs/bleeding_edge/index.html 


Dartium: http://www.dartlang.org/dartium 


Dart Source Code and Bug Tracking: http://code.google.com/p/dart 


HTML5 for Publishers: http://shop.oreilly.com/product/0636920022473.do 
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Big Data TechCon! 


The ) conference for Big Data and IT Professionals 


Discover how to master Big Data from real-world 
practitioners — instructors who work in the trenches 
and can teach you from real-world experience! 


Collect, sort and store massive quantities 
of structured and unstructured data 


Process real-time data pouring 
into your organization 


Master Big Data tools and 
technologies like Hadoop, Map/Reduce, 
NoSQL databases, and more 


Learn HOW TO integrate data-collection technologies with 
analysis and business-analysis tools to produce the kind of 
workable information and reports your organization needs 


Understand HOW TO leverage Big Data to help 
your organization today 


BigData 


TECHCON 


April 8-10, 2013 
Boston, MA 
www.BigDataTechCon.com 


Register Early and SAVE! 


A BZ Media Event 


Big Data TechCon™ is a trademark of BZ Media LLC. 
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Varnish 


Speed Up Your 
Web Site with 


Want your Web site to respond faster and scale better? Learn 
how to use Varnish to make it happen. PABLO GRAZIANO 


Varnish is a program that can 
greatly speed up a Web site while 
reducing the load on the Web 
server. According to Varnish's official 
site, Varnish is a “Web application 
accelerator also known as a caching 
HTTP reverse proxy”. 

When you think about what a 
Web server does at a high level, it 
receives HTTP requests and returns 
HTTP responses. In a perfect world, 
the server would return a response 
immediately without having to do any 
real work. In the real world, however, 
the server may have to do quite a bit 
of work before returning a response 
to the client. Let's first look at how 
a typical Web server handles this, 
and then see what Varnish does to 
improve the situation. 
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Although every server is different, 
a typical Web server will go through 
a potentially long sequence of steps 
to service each request it receives. 
It may start by spawning a new 
process to handle the request. Then, 
it may have to load script files from 
disk, launch an interpreter process 
to interpret and compile those files 
into bytecode and then execute 
that bytecode. Executing the code 
may result in additional work, such 
as performing expensive database 
queries and retrieving more files 
from disk. Multiply this by hundreds 
or thousands of requests, and you 
can see how the server quickly can 
become overloaded, draining system 
resources trying to fulfill requests. 
To make matters worse, many of 


the requests are repeats of recent 
requests, but the server may not have 
a way to remember the responses, so 
it’s sentenced to repeating the same 
painful process from the beginning for 
each request it encounters. 

Things are a little different with 
Varnish in place. For starters, the 
request is received by Varnish instead 
of the Web server. Varnish then will 
look at what’s being requested and 
forward the request to the Web server 
(known as a back end to Varnish). The 
back-end server does its regular work 
and returns a response to Varnish, 
which in turn gives the response to the 
client that sent the original request. 

If that’s all Varnish did, it wouldn't 
be much help. What gives us the 
performance gains is that Varnish can 
store responses from the back end in 
its cache for future use. Varnish quickly 
can serve the next response directly 
from its cache without placing any 
needless load on the back-end server. 
The result is that the load on the back 
end is reduced significantly, response 
times improve, and more requests 
can be served per second. One of the 
things that makes Varnish so fast is 
that it keeps its cache completely in 
memory instead of on disk. This and 
other optimizations allow Varnish to 
process requests at blinding speeds. 
However, because memory typically is 
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more limited than disk, you have to 
size your Varnish cache properly and 
take measures not to cache duplicate 
objects that would waste valuable space. 

Let's install Varnish. I’m going to 
explain how to install it from source, 
but you can install it using your 
distribution’s package manager. The 
latest version of Varnish is 3.0.3, and 
that’s the version | work with here. Be 
aware that the 2.x versions of Varnish 
have some subtle differences in the 
configuration syntax that could trip you 
up. Take a look at the Varnish upgrade 
page on the Web site for a full list of 
the changes between versions 2.x and 
3.x (https://www.varnish-cache.org/ 
docs/3.0/installation/upgrade.html). 

Missing dependencies is one of the 
most common installation problems. 
Check the Varnish installation page 
for the full list of build dependencies 
(https://www.varnish-cache.org/ 
docs/3.0/installation/install.html# 
compiling-varnish-from-source). 

Run the following commands as 
root to download and install the latest 
version of Varnish: 


cd /var/tmp 

wget http://repo.varnish-cache.org/source/varnish-3.0.3.tar.gz 
tar xzf varnish-3.0.3.tar.gz 

cd varnish-3.0.3 

sh autogen.sh 


sh configure 
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make 
make test 


make install 


Varnish is now installed under the 
/usr/local directory. The full path to the 
main binary ts /usr/local/sbin/varnishd, 
and the default configuration file is 
/usr/local/etc/varnish/default.vcl. 

You can start Varnish by running 
the varnishd binary. Before you can 
do that though, you have to tell 
Varnish which back-end server it’s 
caching for. Let’s specify the back 
end in the default.vcl file. Edit the 
default.vcl file as shown below, 
substituting the values for those of 
your Web server: 


backend default { 
snost =. "127 ,0,0,1"5 
.port = "30"; 


Now you can start Varnish with 
this command: 


/usr/local/sbin/varnishd -f /usr/local/etc/varnish/default.vcl 


™-a :6081 -P /var/run/varnish.pid -s malloc,256m 


This will run varnishd as a demon 
and return you to the command 
prompt. One thing worth pointing 
out is that varnishd will launch two 
processes. The first is the manager 
process, and the second is the child 
worker process. If the child process 
dies for whatever reason, the manager 
process will soawn a new process. 

If you installed Varnish from your 
package manager, it may be running 
already. In that case, you can stop it 
first, then use the command above 
to start it manually. Otherwise, the 
options it was started with may differ 
from those in this example. A quick 
way to see if Varnish is running and 


Varnishd Startup Options 


The -f option tells Varnish where your configuration file lives. 


The -a option is the address:port that Varnish will listen on for incoming 


HTTP requests from clients. 


The -P option is the path to the PID file, which will make it easier to stop 


Varnish in a few moments. 


The -s option configures where the cache is kept. In this case, we’re 
using a 256MB memory-resident cache. 
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what options it was given is with the and deliver the response back to the 

pgrep command: client. Let’s submit some simple GET 
requests and see what Varnish does. 

/usr/bin/pgrep -1f varnish First, run these two commands on 


separate terminals: 
Varnish now will relay any requests 
it receives to the back end you /usr/local/bin/varnishlog 
specified, possibly cache the response, /usr/local/bin/varnishstat 


i pablo@tar. — (J (e) 3) 


File Edit View Terminal Help 


root@falcon:~# GET -Used http://localhost:6081/cgi-bin/test 
GET http://localhost:6081/cg1-bin/test 
User-Agent: Lwp-request/5.834 Libwww-perl/5.836 


200 OK 

Cache-Control: private; 

Connection: close 
: Tue, 04 Dec 2012 02:22:11 GMT 
1.1 varnish 


Apache 
Vary: Accept-Encoding 
Content-Type: text/html 
Client-Date: Tue, 04 Dec 2012 02:22:11 GMT 
Client-Peer: 127.0.0.1:6081 
Client-Response-Num: 1 
Client-Transfer-Encoding: chunked 
X-Varnish: 910098091 


root@falcon:~# i 


[2] O:bash* 4:cli- "falcon" 18:22 03-Bec-12 


Figure 1. Varnish Response Headers 
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The following GET command is part 
of the Perl www library (libwww-perl). 
| use it So you can see the response 
headers you get back from Varnish. 

If you don’t have libwww-perl, you 
could use Firefox with the Live HTTP 
Headers extension or another tool of 
your choice: 


GET -Used http://localhost:6081/ 


The options given to the GET 
command aren't important here. 
The important thing is that the URL 
points to the port on which varnishd 
is listening. There are three response 
headers that were added by Varnish. 
They are X-Varnish, Via and Age. 
These headers are useful once you 
know what they are. The X-Varnish 
header will be followed by either one 
or two numbers. The single-number 
version means the response was not in 
Varnish’s cache (miss), and the number 
shown Is the ID Varnish assigned 
to the request. If two numbers are 
shown, it means Varnish found a 
response in its cache (hit). The first is 
the ID of the request, and the second 
is the ID of the request from which 
the cached response was populated. 
The Via header just shows that the 
request went through a proxy. The 
Age header tells you how long the 
response has been cached by Varnish, 
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in seconds. The first response will 
have an Age of 0, and subsequent 
hits will have an incrementing Age 
value. If subsequent responses to the 
same page don’t increment the Age 
header, that means Varnish is not 
caching the response. 

Now let’s look at the varnishstat 
command launched earlier. You should 
see something similar to Figure 2. 

The important lines are cache_hit 
and cache_miss. cache_hits won't 
be shown if you haven't had any hits 
yet. As more requests come in, the 
counters are updates to reflect hits 
and misses. 

Next, let’s look at the varnishlog 
command launched earlier (Figure 3). 

This shows you fairly verbose details 
of the requests and responses that 
have gone through Varnish. The 
documentation on the Varnish Web 
site explains the log output as follows: 


The first column is an arbitrary 
number, it defines the request. 
Lines with the same number are 
part of the same HTTP transaction. 
The second column is the tag of 
the log message. All log entries are 
tagged with a tag indicating what 
sort of activity is being logged. 
Tags starting with Rx indicate 
Varnish is receiving data and Tx 
indicates sending data. The third 


column tell us whether this is data 
coming or going to the client (c) or 
to/from the back end (b). The forth 
column is the data being logged. 


varnishlog has various filtering 
options to help you find what 
you're looking for. | recommend 
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playing around and getting 
comfortable with varnishlog, 
because It will really help you debug 
Varnish. Read the varnishlog(1) man 
page for all the details. Next are 
some simple examples of how to 
filter with varnishlog. 

To view communication between 


(el pablo@tar = Oe Figure 2. 

| File Edit View Terminal Help varnishstat 
JO+02: 16:24 C f 
fHitrate ratio: 1 1 1 omman 
jHi trate avg: 1.0000 1.0000 1.0000 


3212021 
3212101 
3212094 


ooooqooo0o°o 


n_wrk - 


3191145 
3212199 
3212101 

7 
716129277 
167045352 
3212199 
115641582 
12854289 
752150 

34 


oooooooooooo°o]0o 


| 1 : 
|[2]_O:varnishstat* 4:cli- 


392.48 client_conn - 
392.49 client_regq - 
392.48 cache_hit - 
QO. cache_miss - 
backend_conn - Backend conn. success 
backend_toolate - Backend conn. was closed 
backend_recycle - Backend conn. recycles 
fetch_length - Fetch with Length 
n_sess_mem - 
n_objectcore - N struct objectcore 
n_objecthead - N struct objecthead 
n_waltinglist - N struct waitinglist 
n_vbe - N struct vbe 
N worker threads 
n_wrk_create - N worker threads created 
n_wrk_queued - N queued work requests 
n_backend - 
: . m_expired - N expired objects 
135 ; .  nm_lru_moved - 
n_objwrite - 
s_sess - 
s_req 
s_fetch - 
s_hdrbytes - 
s_bodybytes - 
sess closed - Session Closed 
shm_records - 
shm_writes - 
shm_cont - 
shm_cycles - 
backend_req - 
n_vcl - N vel total 
n_vel_avail - 
n_ban - N total active bans 
n_ban_gone - 


Total Sessions 
- Total Requests 
Total fetch 


SHM MTX contention 


Client connections accepted 
Client requests received 
Cache hits 
Cache misses 


N struct sess_mem 


N backends 


N LRU moved objects 
Objects sent with write 


Total header bytes 
Total body bytes 


SHM records 
SHM writes 


SHM cycles through buffer 
Backend requests made 


N vel available 


N total gone bans 
"falcon" 18:25 O03-Dec-12 
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| File Edit View Terminal Help 


OK 

Date: Tue, 04 Dec 2012 02:28:56 GMT 
Server: Apache 

Cache-ControLl: private; 

Vary: Accept-Encoding 
Content-Encoding: gzip 
Content-Length: 63 

Content-Type: text/html 

4(length) cls 0 mklen 1 

63 

default 

127.0.0.1 53321 :6081 

127.0.0.1 53321 1863511864 

GET 

/cegi-bin/test 

HTTP/1.1 

TE: deflate,gzip;q=0.3 

Connection: TE, close 

Host: Localhost:6081 

User-Agent: lwp-request/5.834 Libwww-perl/5.836 
recy Lookup 

hash 

/cegi-bin/test 

localhost: 6081 

hash 

miss fetch 

13 default default 

1863511864 RFC 120 -1 -1 1354588137 0 1354588136 0 0 
fetch deliver 

HTTP/1.1 

OK 

Date: Tue, 04 Dec 2012 02:28:56 GMT 
Server: Apache 

Cache-Control: private; 

Vary: Accept-Encoding 
Content-Encoding: gzip 
Content-Type: text/html 

uF - 63 52 80 80 440 

deliver deliver 

HTTP/1.1 

200 

OK 

Server: Apache 

Cache-Control: private; 

Vary: Accept-Encoding 

Content-Type: text/html 
Transfer-Encoding: chunked 

Date: Tue, 04 Dec 2012 02:28:56 GMT 
X-Varnish: 1863511864 

Age: 0 

Via: 1.1 varnish 

Connection: close 

UD - 63 52 80 80 440 

52 

1863511864 1354588136. 742137194 1354588136. 744198799 0.000128508 0.002001524 


RxResponse 
RxHeader 
RxHeader 
RxHeader 
RxHeader 
RxHeader 
RxHeader 
RxHeader 
Fetch_Body 
Length 
BackendReuse 
SessionOpen 
ReqStart 
RxRequest 
RxURL 
RxProtocol 
RxHeader 
RxHeader 
RxHeader 
RxHeader 
VCL_call 
VCL_call 
Hash 

Hash 
vCL_return 
VCL_call 
Backend 

EVE 

VCL_call 
ObjProtocol 
Ob}Response 
Obj Header 
ObjHeader 
ObjHeader 


ObjHeader 


Ob}Header 


Obj Header 
Gzip 
VCL_call 
TxProtocol 
TxStatus 
TxResponse 
TxHeader 
TxHeader 
TxHeader 
TxHeader 
TxHeader 
TxHeader 
TxHeader 
TxHeader 
TxHeader 
TxHeader 
Gzip 
Length 
ReqEnd 
0.000060081 
11 SessionClose c Connection: close 
11 StatSess 122.0.0.1:3aa2"’6 k ik’ Oo laa 32 


Maan WA AO AN WAAR Ane ON AKA Ta An SS oo Sooo lS oC. 


Figure 3. varnishlog Command 
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Varnish and the client (omitting the 
back end): 


/usr/local/bin/varnishlog -c 

To view communication between 
Varnish and the back end (omitting 
the client): 
/usr/local/bin/varnishlog -b 

To view the headers received 
by Varnish (both the client's 
request headers and the back end’s 
response headers): 


/usr/local/bin/varnishlog -i RxHeader 


Same thing, but limited to just the 
client's request headers: 


/usr/local/bin/varnishlog -c -i RxHeader 


Same thing, but limited to just the 
back end’s response headers: 


/usr/local/bin/varnishlog -b -i RxHeader 


To write all log messages to the 
/var/log/varnish.log file and deemonize: 


/usr/local/bin/varnishlog -Dw /var/log/varnish. log 


To read and display all log messages 
from the /var/log/varnish.log file: 
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/usr/local/bin/varnishlog -r /var/log/varnish. log 


The last two examples demonstrate 
storing your Varnish log to disk. 
Varnish keeps a circular log in memory 
in order to stay fast, but that means 
old log entries are lost unless saved 
to disk. The last two examples above 
demonstrate how to save all log 
messages to a file for later review. 

If you wanted to stop Varnish, you 
could do so with this command: 
kill “cat /var/run/varnish.pid- 

This will send the TERM signal to 
the process whose PID Is stored in 
the /var/run/varnish.pid file. Because 
this is the varnishd manager process, 
Varnish will shut down. 

Now that you know how to start 
and stop Varnish, and examine cache 
hits and misses, the natural question 
to ask is what does Varnish cache, and 
for how long? 

Varnish is conservative with what 
it will cache by default, but you can 
change most of these defaults. It will 
consider only caching GET and HEAD 
requests. It won’t cache a request 
with either a Cookie or Authorization 
header. It won’t cache a response with 
either a Set-Cookie or Vary header. 
One thing Varnish looks at is the 
Cache-Control header. This header 
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is optional, and it may be present 

in the Request or the Response. It 
may contain a list of one or more 
semicolon-separated directives. 

This header is meant to apply 
caching restrictions. However, Varnish 
won't alter its caching behavior 
based on the Cache-Control header, 
with the exception of the max-age 
directive. This directive looks like 
Cache-Control: max-age=n, 
where n is a number. If Varnish 
receives the max-age directive in 
the back end’s response, it will 

use that value to set the cached 


response’s expiration (TTL), in seconds. 


Otherwise, Varnish will set the cached 
response’s TTL expiration to the value 
of its default_ttl parameter, which 
defaults to 120 seconds. 

You'll likely want to change what 
Varnish caches and how long It’s 
cached for—this ts called your caching 


policy. You express your caching policy 
in the default.vcl file by writing VCL. 
VCL stands for Varnish Configuration 
Language, which Is like a very simple 
scripting language specific to Varnish. 
VCL is fully explained in the vcl(7) man 
page, and | recommend reading it. 
Before changing default.vcl, let’s 
think about the process Varnish goes 
through to fulfill an HTTP request. | 
call this the request/response cycle, 
and it all starts when Varnish receives 
a request. Varnish will parse the 
HTTP request and store the details in 
an object known to Varnish simply 
as req. Now Varnish has a decision 
to make based entirely on the req 
object—should it check its cache for 
a match or just forward the request 
to the back end without caching the 
response? If it decides to bypass its 
cache, the only thing left to do is 
forward the request to the back end 


NOTE: Varnish has configuration parameters with sensible defaults. For example, 

the default_tt] parameter defaults to 120 seconds. Configuration parameters are fully 
explained in the varnishd(1) man page. You may want to change some of the default 
parameter values. One way to do that is to launch varnishd by using the -p option. 
This has the downside of having to stop and restart Varnish, which will flush the cache. 
A better way of changing parameters is by using what Varnish calls the management 
interface. The management interface is available only if varnishd was started with the 
-T option. It specifies on what port the management interface should listen. You can 
connect to the management interface with the varnishadm command. Once connected, 
you can query parameters and change their values without having to restart Varnish. 


To learn more, read the man pages for varnishd, varnishadm and varnish-cli. 
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and then forward the response back called vcl_recv. This gets executed 
to the client. However, if it decides after receiving the full client request, 
to check its cache, things get more which is available in the req object. 
interesting. This is called a cache Here you can inspect and make 
lookup, and the result will either changes to the original request via 
be a hit or a miss. A hit means that the req object. You can use the value 
Varnish has a response in its cache for of req to decide how to proceed. The 
the client. A miss means that Varnish return value is how you tell Varnish 
doesn't have a cached response to what to do. I'll put the return values 
send, so the only logical thing to do in parentheses as they are explained. 
is send the request to the back end Here you can tell Varnish to bypass 
and then cache the response it gives the cache and send the back end’s 
before sending it back to the client. response back to the client (pass). 
Now that you have an idea of You also can tell Varnish to check its 
Varnish’s request/response cycle, cache for a match (lookup). 
let’s talk about how to implement Next is the vcl_pass subroutine. If 
your caching policy by changing you returned pass in vcl_recv, this is 


the decisions Varnish makes in 
the process. Varnish has a set of 


subroutines that carry out the process Varnish 


described above. Each of these A 

subroutines performs a different part Subroutines 

of the process, and the return value The Varnish subroutines have default 
from the subroutine is how you tell definitions, which are shown in 
Varnish what to do next. In addition default.vcl. Just because you redefine 
to setting the return values, you can one of these subroutines doesn’t 
inspect and make changes to various mean the default definition will not 
objects within the subroutines. execute. In particular, if you redefine 
These objects represent things like one of the subroutines but don’t 

the request and the response. Each return a value, Varnish will proceed 
subroutine has a default behavior to execute the default subroutine. All 
that can be seen in default.vcl. You the default Varnish subroutines return 
can redefine these subroutines to get a value, so it makes sens that Varnish 
Varnish to behave how you want. uses them as a fallback. 


The first subroutine to look at is 
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where you'll be just before sending 
the request to the back end. You can 
tell Varnish to continue as planned 
(pass) or to restart the cycle at the 
vcl_recv subroutine (restart). 

The vcl_miss and vcl_hit subroutines 
are executed depending on whether 
Varnish found a suitable response in 
the cache. From vcl_miss, your main 
options are to get a response from the 
back-end server and cache it (fetch) 
or to get a response from the back 
end and not cache it (pass). vcl_hit is 
where you'll be if Varnish successfully 
finds a matching response in its cache. 
From vcl_hit, you have the cached 
response available to you in the ob| 
object. You can tell Varnish to send 
the cached response to the client 
(deliver) or have Varnish ignore the 
cached response and return a fresh 
response from the back end (pass). 

The vcl_fetch subroutine is where 
you'll be after getting a fresh response 
from the back end. The response 
will be available to you in the beresp 
object. You either can tell Varnish to 


continue as planned (deliver) or to 
Start over (restart). 

From vcl_deliver, you can finish the 
request/response cycle by delivering 
the response to the client and possibly 
caching it as well (deliver), or you can 
Start over (restart). 

As previously stated, you express 
your caching policy within the 
subroutines in default.vcl. The return 
values tell Varnish what to do next. 
You can base your return values on 
many things, including the values 
held in the request (req) and response 
(resp) objects mentioned earlier. In 
addition to reg and resp, there also 
is a client object representing the 
client, a server object and a beresp 
object representing the back end’s 
response. It’s important to realize 
that not all objects are available in 
all subroutines. It’s also important 
to return one of the allowed return 
values from subroutines. One of 
the hardest things to remember 
when starting out with Varnish is 
which objects are available in which 


Table 1. This table shows which objects are available in each of the subroutines. 


vel_recv 
vcl_pass 
vcl_miss 
vel_hit 


vel_fetch 


vel_deliver 
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subroutines, and what the legal 
return values are. To make it easier, 
I've created a couple reference tables. 
They will help you get up to speed 
quickly by not having to memorize 
everything up front or dig through 
the documentation every time you 
make a change. 


TIP: Be sure to read the full explanation 
of VCL, available subroutines, return 
values and objects in the vcl(7) man page. 


Let's put it all together by looking at 
some examples. 
Normalizing the request’s Host header: 


sub vcl_recv { 
if (req.http.host ~ "4www.example.com") { 


set req.http.host = "example.com"; 


Notice you access the request’s host 
header by using req.http.host. You 
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have full access to all of the request’s 
headers by putting the header name 
after req.http. The ~ operator is the 
match operator. That is followed by 
a regular expression. If you match, 
you then use the set keyword and 
the assignment operator (=) to 
normalize the hostname to simply 
“example.com”. A really good reason 
to normalize the hostname is to 
keep Varnish from caching duplicate 
responses. Varnish looks at the 
hostname and the URL to determine 
if there’s a match, so the hostnames 
should be normalized if possible. 
Here's a snippet from the default 
vcl_recv subroutine: 


sub vcl_recv { 
if (req.request != "GET" && req.request != "HEAD") { 
return (pass) ; 


} 


return (Lookup) ; 


That's a snippet of the default 


Table 2. This table shows valid return values for each of the subroutines. 


restart 


vel_recv 
vcl_pass 
vel_lookup 
vel_miss 
vel_hit 
vel_fetch 


vel_deliver 


| deliver 


| fetch | hit_for_pass 


WWW.LINUXJOURNAL.COM / MARCH 2013 / 121 


INDEPTH 


vcl_recv subroutine. You can see that 
if it’s not a GET or HEAD request, 
Varnish returns pass and won't cache 
the response. If it is a GET or HEAD 
request, it looks it up in the cache. 

Removing request’s Cookies if the 
URL matches: 


sub Vcl_recy { 
if (req.url ~ "*/images") { 
unset req. http. cookie; 


That’s an example from the Varnish 
Web site. It removes cookies from 
the request if the URL starts with 
“/images”. This makes sense when 
you recall that Varnish won't cache 
a request with a cookie. By removing 
the cookie, you allow Varnish to cache 
the response. 

Removing response cookies for 
image files: 


sub vcl_fetch { 
it (req.Url ~ “\.( png) err |jpe) a") 4 
unset beresp.http.set-cookie; 
set beresp.ttl = 1h; 


That's another example from 
Varnish’s Web site. Here you're in the 
vcl_fetch subroutine, which happens 
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after fetching a fresh response from 
the back end. Recall that the response 
is held in the beresp object. Notice that 
here you're accessing both the request 
(req) and the response (beresp). If the 
request is for an image, you remove 
the Set-Cookie header set by the server 
and override the cached response’s 

TTL to one hour. Again, you do this 
because Varnish won't cache responses 
with the Set-Cookie header. 

Now, let’s say you want to add a 
header to the response called X-Hit. 
The value should be 1 for a cache 
hit and O for a miss. The easiest way 
to detect a hit is from within the 
vcl_hit subroutine. Recall that vcl_hit 
will be executed only when a cache 
hit occurs. Ideally, you'd set the 
response header from within vcl_hit, 
but looking at Table 1 in this article, 
you see that neither of the response 
objects (beresp and resp) are available 
within vcl_hit. One way around this 
is to set a temporary header in the 
request, then later set the response 
header. Let’s take a look at how to 
solve this. 

Adding an X-Hit response header: 


sub vcl_hit { 


set req.http.tempheader = "1"; 


sub vcl_miss { 


set req. http.tempheader = "0"; 


sub vcl_deliver { 
set resp.http.X-Hit = "0"; 


if (req.http.tempheader) { 


set resp.http.X-Hit = req.http.tempheader ; 


unset req. http. tempheader ; 


The code in vcl_hit and vcl_miss 
is straightforward—set a value ina 
temporary request header to indicate 
a cache hit or miss. The interesting bit 
is in vcl_deliver. First, | set a default 
value for X-Hit to 0, indicating a miss. 
Next, | detect whether the request’s 
tempheader was set, and if so, set the 
response’s X-Hit header to match the 
temporary header set earlier. | then 
delete the tempheader to keep things 
tidy, and I’m all done. The reason | 
chose the vcl_deliver subroutine Is 
because the response object that will 
be sent back to the client (resp) is 
available only within vcl_deliver. 

Let’s explore a similar solution that 
doesn’t work as expected. 

Adding an X-Hit response header— 
the wrong way: 


sub vcl_hit { 


set req.http.tempheader = "1"; 
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sub vcl_miss { 


set req.http.tempheader = "0"; 


sub vcl_fetch { 
set beresp.http.X-Hit = "0"; 
if (req.http.tempheader) { 
set beresp.http.X-Hit = req.http.tempheader ; 


unset req.http.tempheader ; 


Notice that within vcl_fetch, I’m 
now altering the back end’s response 
(beresp), not the final response sent to 
the client. This code appears to work 
as expected, but it has a major bug. 
What happens is that the first request 
is amiss and fetched from the back 
end, and that response has X-Hit set 
to “O” then it’s cached. Subsequent 
requests result in a cache hit and never 
enter the vcl_fetch subroutine. The 
result is that all cache hits continue 
having X-Hit set to “O”. These are the 
types of mistakes to look out for when 
working with Varnish. 

The easiest way to avoid these 
mistakes is to keep those reference 
tables handy; remember when each 
subroutine is executed in Varnish’s 
workflow, and always test the results. 

Let’s look at a simple way to tell 
Varnish to cache everything for 
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one hour. This is shown only as an 
example and isn’t recommended for a 
real server. 

Cache all responses for one hour: 


sub Vcl_recy { 
return (Lookup) ; 


sub vel fetch 4 
set beresp.ttl = 1h; 
return (deliver) ; 


Here, I’m overriding two default 
subroutines with my own. If | hadn't 
returned “deliver” from vcl_fetch, 
Varnish still would have executed its 
default vcl_fetch subroutine looking 
for a return value, and this would not 
have worked as expected. 

Once you get Varnish to implement 
your caching policy, you should run 
some benchmarks to see if there is 
any improvement. The benchmarking 
tool | use here is the Apache 
benchmark tool, known as ab. You 
can install this tool as part of the 
Apache Web server or as a separate 
package—depending on your system's 
package manager. You can read 
about the various options available to 
ab in either the man page or at the 
Apache Web site (httpd.apache.org/ 
docs/2.4/programs/ab.html). 
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In the benchmark examples 
below, | have a stock Apache 2.2 
installation listening on port 80, 
and Varnish listening on port 6081. 
The page I’m testing is a very basic 
Perl CGI script | wrote that just 
outputs a one-liner HTML page. It’s 
important to benchmark the same 
URL against both the Web server and 
Varnish so you can make a direct 
comparison. | run the benchmark 
from the same machine that Apache 
and Varnish are running on in 
order to eliminate the network as 
a factor. The ab options | use are 
fairly straightforward. Feel free to 
experiment with different ab options 
and see what happens. 

Let's start with 1000 total requests 
(-n 1000) and a concurrency of 1 (-c 1). 

Benchmarking Apache with ab: 


ab -c 1 -n 1000 http://localhost/cgi-bin/test 
Benchmarking Varnish with ab: 
ab -c 1 -n 1000 http://localhost:6081/cgi-bin/test 


As you can see, the ab command 
provides a lot of useful output. 
The metrics I’m looking at here are 
“Time per request” and “Requests 
per second” (rps). You can see that 
Apache came in at just over 1ms per 
request (780 rps), while Varnish came 


File Edit View Terminal Help 


root@localhost # ab -c 1 -n 1000 http://localhost/cgi-bin/test 

This 1s ApacheBench, Version 2.3 <$Revision: 655654 $> 

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ 
Licensed to The Apache Software Foundation, http://www.apache.org/ 


Benchmarking localhost (be patient) 
eted 100 requests 
eted 200 requests 
leted 300 requests 
eted 400 requests 
eted 500 requests 
eted 600 requests 
eted 700 requests 
eted 800 requests 
eted 900 requests 
eted 1000 requests 
Finished 1000 requests 


Server Software: Apache 
Server Hostname: localhost 
Server Port: t=10] 


Document Path: /cgi-bin/test 
Document Length: 52 bytes 


Concurrency Level: 1 

Time taken for tests: 1.281 seconds 

Complete requests: 1000 

Failed requests: io) 

Write errors: 10] 

Total transferred: 216060 bytes 

HTML transferred: 52000 bytes 
per second: 780.90 [#/sec] (mean) 
request: 1.281 [ms] (mean) 
request: 1.281 [ms] (mean, across all concurrent requests) 
rate: 164.77 [Kbytes/sec] received 


Connection Times (ms) 

min mean[+/-sd] median max 
Connect: 10) 10] 19] 10} 
Processing: 1 1 
Waiting: 1 1 
Total: 1 1 


0. 
oO. 
0. 
oO. 


1 2 
1 2 
1 2 


Percentage of the requests served within a certain time (ms) 
1 


2 (longest request) 
root@localhost # | 


Figure 4. Output from ab Command (Apache) 


in at 0.1ms (7336 rps)—nearly ten with various options to get a feel for 
times faster than Apache. This shows performance—particularly by changing 
that Varnish is faster, at least based the concurrency values and seeing 

on the current setup and isolated what impact that has on your system. 
testing. It’s a good idea to run ab The goal is to not only improve 
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File Edit View Terminal Help 

root@localhost # ab -c 1 -n 1000 http://lLocalhost:6081/cgi-bin/test 

This 1s ApacheBench, Version 2.3 <$Revision: 655654 $> 

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ 
Licensed to The Apache Software Foundation, http://www.apache.org/ 


Benchmarking localhost (be patient) 
leted 100 requests 
leted 200 requests 
leted 300 requests 
leted 400 requests 
leted 500 requests 
leted 600 requests 
leted 700 requests 
leted 800 requests 
leted 900 requests 
leted 1000 requests 
Finished 1000 requests 


Server Software: Apache 
Server Hostname: localhost 
Server Port: 6081 


Document Path: /cgi-bin/test 
Document Length: 52 bytes 


Concurrency Level: i 

Time taken for tests: 0.136 seconds 

Complete requests: 1000 

Failed requests: (0) 

Write errors: 10] 

Total transferred: 273990 bytes 

HTML transferred: 52000 bytes 
per second: 7336.22 [#/sec] (mean) 
request: 0.136 [ms] (mean) 
request: 0.136 [ms] (mean, across all concurrent requests) 
rate: 1962.94 [Kbytes/sec] received 


Connection Times (ms) 

min mean[+/-sd] median max 
Connect: 10] 10] F 10] 10] 
Processing: 


ic} . io} 2 
ic} . 10} 2 
io} 1°} 2 


Percentage of the requests served within a certain time (ms) 
SO% 10] 


6 2 (longest request) 
root@localhost # | 


Figure 5. Output from ab Command (Varnish) 


response times, but also to do so with ~—= system performance are the load 
as little impact on system resources average and the %iowait. The load 
as possible. Let’s compare how a average can be seen with the top 
orolonged traffic surge affects system utility, and the %iowait can be seen 
resources. Two good measures of with the iostat command. You're 
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System Load and “%iowait 


System load is a measure of how much load is being placed on your CPU(s). 
As a general rule, you want the number to stay below 1.0 per CPU or core on 
your system. That means if you have a four-core system as in the machine I’m 
benchmarking here, you want your system’s load to stay below 4.0. 


%iowait is a measure of the percentage of CPU time spent waiting on input/output. 
A high %iowait indicates your system is disk-bound, performing many disk i/o 
operations causing the system to slow down. For example, if your server had to 
retrieve 100 files or more for each request, it likely would cause the %iowait time 
to go up very high indicating that the disk is a bottleneck. 


going to want to keep an eye on both 
top and iostat during the prolonged 
load test to see how the numbers 
change. Let's fire up top and iostat, 
each on separate terminals. 

Starting iostat with a two-second 
update interval: 


iostat -c 2 
Starting top: 
/usr/bin/top 


Now you're ready to run the 
benchmark. You want ab to run 
long enough to see the impact on 
system performance. This typically 
means anywhere from one minute 
to ten minutes. Let’s re-run ab with 


a lot more total requests and a 
higher concurrency. 
Load testing Apache with ab: 


ab -c 50 -n 100000 http://localhost/cgi-bin/test 
Load testing Varnish with ab: 
ab -c 50 -n 1000000 http://localhost:6081/cgi-bin/test 


First let's compare response times. 
Although you can’t see it in the 
screenshots, which were taken just 
before ab finished, Apache came 
in at 23ms per request (2097 rps), 
and Varnish clocked in at 4ms per 
request (12099 rps). The most 
drastic difference can be seen in the 
load averages in top. While Apache 
brought the system load all the way 
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View Terminal Help 


user 
37.16 


Suser 
37.09 


%user 
38.65 


%user 
38.00 


%user 
36.09 


%nice %system %iowait 
0.00 61.85 0.00 


Ssnice ‘system %s1owa1lt 
0.00 62.66 0.00 


Ssnice “system %lowa1lt 
0.00 60.60 0.00 


Ssnice “system %slowa1lt 
0.00 61.50 0.00 


%nice %ssystem %lowait 
0.00 62.91 0.00 


- 16:51:21 up 154 days, 9:09, 1 user, 


264 

37. Os 
3103920k 
12109808k 


9349 root 


34 222 
0.0% 0.7% 
2175604k 


20 O 8228 4952 1512 R 


load 
10] 
0.0% 


928316k 
45240k 12064568k 


2 


1 
2 
2 
2 
2 
2 
2 
2 
2 
2 


0.2 


average: 12.38, 4.33, 1.55 


8 
0.0% 3.8% 
553236k 
1246636k 


0:08.82 ab 


root@falcon:~# ab -c 50 -n 100000 http://localhost/cgi-bin/test 
This is ApacheBench, Version 2.3 <$Revision: 655654 $> 

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ 
Licensed to The Apache Software Foundation, http://www.apache.org/ 


Benchmarking localhost (be patient) 


10000 
20000 
30000 
40000 
50000 
60000 
70000 
80000 
90000 


requests 
requests 
requests 
requests 
requests 
requests 
requests 
requests 
requests 


0.0% 


Figure 6. System Load Impact of Traffic Surge on Apache 


up to 12, Varnish kept the system 
load near O at 0.4. | did have to wait 
several minutes for the machine's load 
averages to go back down after the 
Apache load test before load testing 
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Varnish. It’s also best to run these 
tests on a non-production system that 
is mostly idle. 

Although everyone's servers 
and Web sites have different 


=] 
File Edit View Terminal Help 


Ssuser Snice ‘%ssystem %slowa1lt 
17.37 0.00 32.51 0.00 


%suser %nice ‘%system %lowa1lt 
19.00 0.00 26.00 0.00 


S%user ‘nice “system %lowait 
19.40 0.00 28.86 0.00 


Ssuser Snice ‘ssystem %lowa1lt 
1712 0.00 30.77 0.00 


%suser %nice ‘system %lowa1lt 
18.36 0.00 26.30 0.00 


- 17:27:17 up 154 days, 9:45, 1 user, load average: 0.44, 0.15, 0.20 
177 2 175 10] 19] 
18.6% 21.1% 0.0% S52. 1% 0.0% 0.0% 8.3% 0.0% 
3103920k 2190000k 913920k 556004k 
12109808k 45240k 12064568k 1247940k 


123 
13665 root 20 0 36352 30m 1516 R 72 1.0 0:52.94 ab 
1 


root@falcon:~# ab -c 50 -n 1000000 http://Localhost:6081/cgi-bin/test 
This 1s ApacheBench, Version 2.3 <$Revision: 655654 $> 

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ 
Licensed to The Apache Software Foundation, http: //www.apache.org/ 


Benchmarking localhost (be patient) 
100000 requests 
200000 requests 
300000 requests 
400000 requests 
500000 requests 
600000 requests 
700000 requests 
800000 requests 
900000 requests 


Figure 7. System Load Impact of Traffic Surge on Varnish 


requirements and configurations, 
Varnish may be able to improve your 
site’s performance drastically while 
simultaneously reducing the load on 
the server.m 


Pablo Graziano has owned and operated a Linux consulting 
company in the Seattle area for more than ten years. He's also an 
avid Perl programmer and dabbles in Java. 
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DOC SEARLS 


Android for 
Independence 


Building the next personal revolution on the dominant platform 
we already have. 


AT&T contract runs out. The plan after 
that is to replace them with Androids. 


t some point in the early 
2000s, | got my wife 
a Nokia phone with a 


keyboard, so we could text each 
other. It was a great little phone, 
not hard to use or understand, 
but she texted me only once with 
it, to send the word “no”. Then, 
in late 2007, not long after the 
iPhone came out, she told me she 
wanted one. Why? “Because | can 
work with it.” So we got her one, 
and she worked it like a chef with 
a collection of Wusthofs. A few 
months later, | got an iPhone 3G. 
She immediately schooled me on 
how to use the thing, and she still 
knows more about texting and 
other essential iPhone apps than | 
do. After the iPhone 4 came out in 
mid-2010, we traded up to those, 


and that’s where we'll stay until our 
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We've also been using Androids all 
along. We got a Nexus One when it 
came out in 2010, and we've gone 
through a series of HTCs since then. 
Our teenage son’s phone Is an HTC 
Detail, which is an unlocked version 
of the HTC Evo Shift 4G. In our now 
ample experience, the HTC Detail is 
crap. In fact, every Android phone 
we've had has failed. But none of 
those failings has cost us faith in 
the platform. Lately, that faith has 
been invested in Samsung's line of 
smartphones and pads, especially the 
Note, after envying one over dinner 
one evening with friends. My wife 
took one look at it and said, “I can 
work with that.” This is an ominous 
sign for Apple. 

So is “Samsung vs. Apple: Losing My 


Look back up the road of history, and it’s clear 
that Apple’s foot has been off the gas ever since 


Steve Jobs died. 


Religion” (http://www.mediapost.com/ 
publications/article/188716/ 
samsung-vs-apple-losing-my- 
religion.html#axzz2Gwynhd3W), 

a column by Barbara Lippert in 
MediaPost in December 2012. 
Barbara, who says she belongs to 
the “cult of Cupertino”, heaps 
praise on Samsung for inspired 
marketing work at a time when 
Apple’s marketing seems to be 
going through the motions. But her 
most damning remark is not about 
Apple's marketing, but about its 
flagship device, the iPhone 5. She 
called it “a bit of a '’meh’”. 

Let's face it. The iPhone 5 is a 
stretched iPhone 4s, which Is an 
iPhone 4 with sprinkles. The iPhone 4 
came out in mid-2010, so the whole 
line is pretty old in mobile phone 
years. There is nothing “gotta have” 
about the iPhone 5, and there's at 
least one big deal-killer: Apple’s Maps 
app, which replaces the Google- 
based one that had come standard 
with all iPhones and iPads up until 
iOS 6, which is what the iPhone 5 
runs. The Maps app was a huge fail 


when it came out in September 2012, 
and it’s not noticeably better (to 

me, at least) in January 2013, when 
the app still showed no subways in 
New York, Boston, Paris or London. 
Its traffic reporting is limited to a 
few tiny red dotted lines that are 
easy to miss and appear only on 
major roads. Google finally came to 
Apple’s rescue in December with an 
outstanding Maps app for iPhone 
(and, passive-aggressively, not for 
iPad). But, with monarchical self- 
regard, Apple continued bringing up 
its broken Maps app for searches off 
linked addresses in Mail, Calendar and 
Contact. Just today, the Apple Maps 
app failed to find my eye doctor's 
address on a main street in Santa 
Barbara. So | copied the address from 
the calendar and pasted it into the 
Google Maps app, which gave me 
three alternate ways of getting there, 
all with degrees of traffic listed. 

Look back up the road of history, 
and it’s clear that Apple’s foot has 
been off the gas ever since Steve 
Jobs died. Nearly every new product 
since then has been an incremental 
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advance of an old one. The only 
notable exception was the iPad Mini, 
which should have come out a year 
or two earlier and still lacks the 

retina screen featured on the current- 
generation iPad. At this stage, Apple 
has two advantages that no other 
tech-gear maker can match: retail 
stores and customer service. But those 
advantages are limited mostly to first- 
world cities and suburbs. Elsewhere, 
Apple’s mobile devices are bling for 
the rich. And even for those people, 
Apple's advantages will matter less 
and less as the company’s mobile 
products fall farther and farther 
behind the many curves being paved 
by competing devices out in the open 
marketplace. This is too bad, because 
the horizontally driven open world 
needs to see what verticalizing closed- 
system makers like Apple are up to. In 
The Intention Economy, | explain the 
symbiosis between the two: 


The smartphone business was 
invented by Nokia and RIM 
around the turn of the millennium 
and then royally disrupted by 
Apple and Google a few years 
later. Today, Apple and Google 
define the smartphone business, 
together, though not always in 
direct competition. It is important 
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to understand how this works, 
because the two companies’ 
directions are orthogonal: ninety 
degrees off from each other. And 
because they do that, the market 
for both—and for everybody else as 
well—is huge. 


Apple’s punch to the smartphone 
market was vertical. It came 

up from below like a volcano 
and went straight toward the 
sky. With the iPhone, Apple 
showed how much invention 

and innovation the old original 
equipment manufacturer 
(OEM)-operator marriage had 
locked out of the smartphone 
marketplace, completely redefining 
the smartphone as a pocket 
computer that also worked as a 
ohone. iPhones were beautiful, 
easy to use, and open to a Zillion 
applications that were easy for 
programmers to write and for 
users to install. 


The Google punch was horizontal. 
It came from the side, spreading 
its open Android platform toward 
the far horizons. As a platform, 
Android supported everything 
Apple’s iOS did—and more, 
because it was open to anybody, 


making it more like geology than a 
foundation. The old cartels could 
still build vertical silos on Android, 
but anybody could build just about 
anything, anywhere. 


So, while Apple shows how high 
one can pile up features and 
services inside one big beautiful 
high-rise of a silo, Google provides 
a way not only to match or beat 
Apple's portfolio, but to show 
how broad and rich the open 
marketplace can be. 


We need innovation in both 
directions, but we can't see how 
complementary these vectors 

are if we cast the companies 
innovating in both directions as 
competitors for just one space. 
So, while it’s true that Apple 
phones compete with Android- 
based phones straight up, it’s also 
true that Apple makes phones 
and Google doesn’t. And, while 
it’s true that iOS and Android 
compete for developers, iOS runs 
only on Apple devices; there is no 
limit to the number and variety 
of devices that run on Android. In 
the larger picture here, Apple and 
Google are stretching the market 
in orthogonal directions. The 


EOF 


result is a bigger marketplace for 
both and for everybody else who 
depends on smartphones. 


| wrote that in late 2011, in faith 
that Apple would continue to break 
new ground in the vertical direction. 
Sadly, it hasn’t (or barely). But ground 
is being broken in all directions 
horizontally (as well as some vertical 
ones) on Android. 

The direction that interests 
me most Is toward personal 
independence, both from and to. 
We need independence from phone 
companies, which continue to 
contain what we imagine can be 
done with connected mobile devices. 
And we need independence to do 
what we please, as free and fully 
empowered human beings. 

Toward both ends on the hardware 
side, | am encouraged by Ubuntu for 
Phones (http://www.ubuntu.com/ 
devices/phone) and Ubuntu for 
Android (http://www.ubuntu.com/ 
devices/android), even though 
both are being pitched to corporate 
developers. | also like seeing 
custom ROMs for Google’s Nexus 
7 (http://reviews.cnet.co.uk/ipad- 
and-tablets/top-six-custom-roms- 
for-the-google-nexus-7-50009354). 
The reason there's an after-market for 
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Nexus hardware is that Google designed 
the line to be open and generative 
from the start. As the copy boasts, 
it’s “unlocked and contract free” 
(http://www.google.com/nexus/4). 

On the software side, | want to see 
tools that give each of us ways to 
take control of the Internet of Things 
and of APl-based services. For that, | 
believe we need to be able to do our 
own programming, in the simplest and 
most uncomplicated ways possible. In 
fact, | see personal programming as 
the latest in a series of revolutions in 
which individuals have gained huge 
advances in power. In each of these 
following cases, individuals could do 
far more than could companies and 
large organizations: 


1. Computing, thanks to the PC. 


2. Communications, thanks to 
the Internet. 


3. Portable computing and 
communications, thanks 


the smartphone. 


4. Programming, with...well, that’s 
what's next. Read on. 


We need to be able to program stuff 
in our own lives and in how we interact 
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with two other domains—and to do so 
independently, outside the control of 
any centralized entity or service: 


1. The Internet of Things. 


2. The portfolio of APl-based services 
that are what large organizations 
need to become, whether they 
like it or not. 


It’s very early in the future that will 
grow here, but there are some early 
efforts that should invite our interest. 

The best known (and funded, far as 
| know) is IFTTT (https://ifttt.com), 
which stands for “if this, then that”. 
It lets you make connections between 
“channels” that are actually Web 
services, through those services’ 
APIs. “Each Channel has its own 
Triggers and Actions”, IFTTT 
explains (https://ifttt.com/wtf): 


The this part of a Recipe is a 
Trigger. Some example Triggers 
are “I'm tagged in a photo on 
Facebook” or “I check in on 
Foursquare.” The that part of 

a Recipe is an Action. Some 
example Actions are “send me 

a text message” or “create a 
status message on Facebook”. 
Pieces of data from a Trigger are 


called Ingredients. For example, 
the Ingredients of an Email 
Trigger could be: subject, body, 
attachment, received date and the 
sender's address. Personal Recipes 
are a combination of a Trigger 
and an Action from your active 
Channels. Personal Recipes look 
like this: “if Any new photo by 
(your handle) then Add file from 
URL to (your handle’s) Dropbox.” 


Less well-known, but no 
less interesting, is on{x} 
(https://www.onx.ms/#!landingPage) 
a Microsoft effort targeted (yes) 
at Android devices. Still in beta, it 
“lets you control and extend the 
capabilities of your Android phone 
using a JavaScript API to remotely 
program it”. It currently requires a 
Facebook login, which annoys me, 
but at this early stage, I'll forgive it. 

Last (but far from least) is the 
growing suite of tools, concepts and 
services that have been rolling out of 
Kynetx for the last several years. The 
open-source side Is the language KRL 
and a rules engine (https://github.com/ 
kre/Kinetic-Rules-Engine). Phil Windley 
(http://www.windley.com) is the 
Linus of both, and he describes KRL 
this way: “KRL is a language that is 
designed to help programmers build 
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and understand distributed, persistent 
data objects (PDOs) that live in 
the cloud and interact, primarily, 
through events. Collections of these 
PDOs form what I've frequently 
referred to as a personal cloud” 
(http://www.windley.com/ 
archives/2012/12/programing_ 
the_cloud_with_persistent_data_ 
objects.shtml). These clouds can be 
hosted by one’s self or at a service. 
They are, as Phil describes them, 
operating systems of one’s own, with 
a “core set of services around identity, 
data and communications, as well 
as a programming model” 
(http://www.windley.com/archives/ 
2012/07/a_road_map_for_the_ 
personal_cloud_operating_system. 
shtml). Early applications of this model 
have me more excited than I’ve been 
in years. My wife too, and she’s no 
techie. Take a look at SquareTag 
(http://www.squaretag.com), 
for example. 

KuppingerCole 
(http://www.kuppingercole.com) 
has a term | like for the category being 
built out here: Life Management Platforms 
(http://www.kuppingercole.com/ 
report/advisorylifemanagement 
platforms7060813412). Hey, better 
to manage our own lives in the 
networked world than to be managed 
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by the likes of Apple, mobile-phone 
companies or Google (which, let's 
remember, is an advertising company 
and takes advantage of its position 
with Android to watch much of 
what we’re doing). 

Given Android’s overwhelming and 
growing success as a smartphone 
and tablet platform, it’s obviously 
the bulls-eye toward which this kind 
of development should be targeted. 
Yes, it's not as open as many of us 
would like. For example, the latest 
SDK requires agreement to a nonfree 
license—though reportedly “to give 
Google the power to enforce the free 
software licenses of the components, 
if a third-party tries to break them” 
(http://code.paulk.fr/article0008/ 
what-s-up-with-the-android-sdk). 
But, building on a majority OS is a 
nice place to start, especially when the 
market we're addressing Is everybody. 

I'm curious to see if, and how, 
readers dig some of the software 
developments | visit above—and, better 
yet, how nontechnical friends and 
spouses take to them. Listen for the 
magic line: “I can work with that.” 


Doc Searls is Senior Editor of Linux Journal. He is also a fellow 
with the Berkman Center for Internet and Society at Harvard 
University and the Center for Information Technology and 
Society at UC Santa Barbara. 
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